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the  four  instructional  modules  of  Narrative,  Procedural  Directions,  Schematics,  and  Forms.  Strand  H was  designed  to 
improve  basic  reading  and  thinking  skills,  in  addition  to  basic  job  concepts  and  vocabulary.  Strand  D source  material 
was  comprised  of  a series  of  passages  written  specifically  for  the  JORP.  Each  package  dealt  with  a different  job 
content  area  and  was  written  at  a lower  level  of  difficulty  than  the  typical  job  reading  materials.  The  instructional 
procedures  for  Strand  11  emphasized  direct  teacher  instruction/  group  activities,  discussion,  and  oral  and  graphic 
responses  from  the  students.  Tests  were  developed  for  student  mastery  and  feedback  in  the  Strand  I work.  Since 
Strand  U student  responses  were  neither  right  nor  wrong,  they  were  judged  by  the  individual  student,  his  peers,  and 
the  instructor  in  terms  of  appropriateness  to  the  task  at  hand.1  The  JORP  prototype  program  was  field  tested  during 
1976  at  Travis  AFB,  California.  Data  generated  by  this  study  indicated  that  there  was  a significant  improvement  in 
job-specific  JORP  test  scores.  Overall,  the  study  showed  the  JORP  to  be  a valuable  and  feasible  approach  to 
job-specific  reading  training  in  die  Air  Force. 
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SUMMARY 


The  Air  Force  has  been  successful  at  recruiting  personnel  with 
better  than  average  abilities  and  literacy  skills  in  past  years. 

However,  two  considerations  are  worth  noting.  First,  there  have 
been  occasions  during  which  large  surges  of  marginal  ability  persons 
have  been  introduced  into  the  Air  Force  training  system.  Second, 
there  are  indications  that  accessions  of  Air  Force  recruits  in  a non- 
draft environment  may  periodically  result  in  more  personnel  with  low- 
er abilities  and  literacy  skills. 

The  Ail  Force  has  defined  its  literacy  problem  in  terms  of  the 
gap  between  the  reading  demands  of  training  and  job  materials  and  the 
reading  skills  possessed  by  the  personnel  who  use  those  written  mater- 
ials. The  two-pronged  approach  currently  used  to  address  this  problem 
includes  reducing  the  difficulty  of  the  material  on  the  one  hand,  and 
increasing  the  literacy  skills  of  the  individual  on  the  other.  How- 
ever, each  approach  has  its  limitations.  The  strategy  of  reducing  the 
reading  difficulty  of  the  written  materials  can  only  be  carried  so  far 
without  causing  a distortion  in  the  meaning  and  substance  of  the  printed 
text.  The  strategy  of  increasing  the  general  reading  level  of  the  air- 
men normally  does  not  result  in  sufficient  improvement  in  job-related 
reading  skills  to  permit  successful  completion  of  the  training  or  job 
task. 


In  order  to  respond  more  fully  to  the  current  literacy  problems 
in  the  Air  Force,  the  present  study  was  undertaken.  The  purpose  of 
this  study  was  to  develop  and  implement  a prototype  Job-Oriented  Read- 
ing Program  (JORP)  which  stressed  the  acquisition  and  development  of 
job-related  reading  skills  for  Air  Force  personnel.  The  two  major  ob- 
jectives were: 

1.  To  determine  the  feasibility  of  using  a job-related 
approach  to  reading  Instruction  with  airmen  in  the 
Air  Force  training  system. 

2.  To  test  the  effectiveness  of  this  approach  in  an  oper- 
ational setting  for  improving  airmen's  performance  in 
using  job-related  reading  materials. 

The  development  of  the  JORP  drew  upon  the  experiences  of  a somewhat 
similar  effort  in  the  Army  in  functional  literacy  (FLIT).  With  that  as 
a starting  point,  the  special  design  of  the  JORP  for  the  Air  Force  in- 
cluded the  following  design  requirements:  the  reading  grade  level  (RGL) 


of  JORP  was  set  at  9.0;  student  Input  RGL  was  from  6.0  to  8.9;  JORP 
training  was  to  be  Integrated  In  the  duty  day  of  the  permanent  party 
personnel;  time  available  for  training  was  2%  hours  per  day  for  five 
days  a week  for  six  weeks. 

The  two  Air  Force  job  career  clusters  chosen  for  this  effort  were 
maintenance  and  non-maintenance  areas.  The  five  Air  Force  Specialty 
Codes  (AFSC)  selected  for  the  maintenance  cluster  were  421X2  - Aircraft 
Pneudraulic  Repairman,  431X1A  - Aircraft  Maintenance  Specialist,  431X1C  - 
Aircraft  Maintenance  Specialist,  431X1F  - Aircraft  Maintenance  Specialist, 
and  431X1E  - Aircraft  Maintenance  Specialist.  For  the  nonmaintenance 
JORP  cluster  the  following  three  AFSCs  were  selected:  702X0  - Adminis- 
tration Specialist,  645X0  - Inventory  Management  Specialist,  and  647X0  - 
Materiel  Facilities  Specialist. 

IVo  instructional  strands  were  developed.  In  Strand  I the  students 
utilized  their  existing  literacy  skills  to  practice  locating,  extracting, 
analyzing,  and  comprehending  job-related  information.  The  source  material 
for  Strand  I was  excerpted  directly  from  job  and  training  manuals.  In- 
structional techniques  emphasized  individual  practice,  self-pacing,  and 
written  responses  utilizing  worksheets  and  tests.  The  four  instructional 
modules  were  narrative,  procedural  directions,  schematics,  and  forms. 

Strand  II  was  designed  to  improve  basic  reading  and  thinking  skills, 
via  basic  job  concepts  and  vocabulary.  Strand  II  source  material  was 
comprised  of  a series  of  passages  written  specifically  for  the  JORP. 

The  passages  each  dealt  with  a different  job  content  area  and  were  writ- 
ten at  a lower  level  of  difficulty  than  the  typical  job  reading  materials. 

The  instructional  procedures  for  Strand  II  emphasized  direct  teacher  in- 
struction, group  activities,  discussion,  and  oral  and  graphic  responses 
from  the  students. 

\ I 

Tests  were  developed  for  student  mastery  and  feedback.  These  were 
most  heavily  used  in  the  Strand  I work.  For  Strand  II  work,  the  responses 
of  the  student  were  neither  right  nor  wrong,  in  the  sense  that  there  was 
only  one  correct  answer.  Rather,  the  responses  were  judged  by  the  in- 
dividual student,  his  peers,  and  the  instructor  in  terms  of  appropriateness 
to  tne  task  at  hand. 

The  JORP  prototype  program  was  field  tested  during  1976  at  Travis 
APB,  California.  There  were  85  male  and  8 female  students,  with  a mean 
age  of  23  years  and  a mean  entering  RGL  of  9.24.  Forty  five  of  the 
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students  entered  with  RGL  below  9.0.  All  students  were  assigned  to 
Travis  AFB  on  a permanent  party  basis. 

For  the  Strand  I instruction,  the  combined  results  (average) 
from  the  four  instructional  modules  Indicated  that  8%  of  the  students 
passed  the  pretest  for  the  instructional  module  and  advanced  immediately 
to  the  next  module;  60%  failed  the  pretest  and  later  passed  the  post 
test;  and  32%  failed  the  pretest  and  subsequently  failed  the  post  test. 
The  overall  average  training  effectiveness  was  64%.  It  should  be  noted 
that  those  students  who  did  fail  both  the  pre-  and  post  tests  were 
moved  on  to  other  training  modules  despite  the  failing  post  test  score. 
This  was  done  to  insure  that  all  students  were  exposed  to  all  training 
modules.  Analysis  indicated  that  had  the  instructional  time  been  longer, 
fewer  students  would  have  failed  the  post  tests.  Of  those  students  who 
failed  the  pretests,  38%  failed  because  of  inaccuracy  and  46%  failed  be- 
cause of 'inaccuracy  and  slow  work  (time).  Thus,  it  seemed  reasonable  to 
conclude  that,  although  most  students  do  enter  the  JORP  with  some  ability 
to  perform  reading  tasks,  they  do  show  a need  for  additional  training  on 
the  fundamental  skills  which  are  taught  in  the  JORP. 

The  finishing  RGL  of  the  stu-.ents  was  9.65.  This  represents  a gain 
of  .41  in  RGL.  In  terms  of  overall  reading  ability,  this  gain  is  not 
statistically  significant.  However,  on  the  job-specific  JORP  test,  the 
entry  score  was  33. 4 r d tne  exit  s :ore  was  49.5.  This  gain  of  16.1 
points  was  signifies.  (p>'  .001)  ' aus,  important  gains  were  made  in 

job-specific  reading  1:  «v  . vcy  ski!1'.  These  results  should  be  viewed 
in  light  of  the  fat;  Itu  t 48  of  the  students  already  had  reading  ability 
at  or  above  the  ? t.L  This  fact  tends  to  reduce  the  apparent  impact 

of  the  JORP  progra.u  /ith  respect  to  RGL,  even  though  the  more  able  stu- 
dents seemed  to  'r.pto  in  their  job-specific  skills  as  measured  by  the 
JORP  Test. 

The  field  test  generated  evidence  that  suggests  that  the  job  rele- 
vance of  the  content  of  the  literacy  training  curriculum  is  an  Important 
and  potent  variable  which  will  influence  the  effectiveness  of  the  train- 
ing program.  Overall,  the  study  showed  the  JORP  to  be  a valuable  and 
feasible  training  plan  for  Job-specific  reading  training  in  the  Air  Force. 
If  the  JORP  is  to  be  implemented  on  an  operational  basis,  it  is  suggested 
that  several  additional  clusters  be  developed.  Specifically,  it  is 
recommended  that:  (1)  the  present  nonmaintenance  cluster  be  split  into 
a 70/73  administrative  cluster  and  a 64  supply  cluster,  and  (2)  three 
additional  clusters  be  developed  to  cover  the  54/55  civil  engineering, 

81  security  police,  and  60  transportation  areas.  This  would  provide  a 
total  of  six  clusters  for  full  JORP  implementation,  and  would  address 
most  of  the  problem  readers  in  the  Air  Force.  The  currently  available 
maintenance  and  nonmaintenance  JORP  clusters  could  be  used  until  these 
more  specific  additional  clusters  are  developed. 
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A JOB-ORIENTED  READING  PROGRAM 
FOR  THE  AIR  FORCE:  DEVELOPMENT  & FIELD  EVALUATION 

Chapter  1 

INTRODUCTION 


Technological  change  places  a premium  on  fundamental  information 
processing  skills,  especially  language  and  literacy.  In  highly  complex 
and  technology-based  organizations  like  the  Air  Force,  the  literacy 
skills  of  personnel  contribute  directly  to  the  capability  to  function 
effectively  and  efficiently  in  accomplishing  the  mission. 

In  the  past,  the  Air  Force  has  been  successful  at  recruiting  per- 
sonnel with  better  than  average  abilities  and  literacy  skills.  However, 
there  have  been  occasions  (e.g..  Project  100,000)  during  which  large  surg- 
es of  marginal  ability  recruits  have  been  Introduced  into  the  Air  Force 
training  system.  In  addition,  there  are  indications  that  accessions  of 
Air  Force  recruits  in  a nondraft  environment  will  periodically  result  in 
personnel  with  lower  abilities  and  literacy  skills  (Vitola  & Valentine, 
1970).  This  result  could  become  especially  acute  when  the  U.S.  economy 
is  robust  and  civilian  jobs  are  plentiful  for  higher  ability  personnel. 
Most  recently,  attendees  at  a World-Wide  Air  Force  On-The-Job-Training 
(OJT)  Conference  (January  1974)  expressed  concern  over  the’  existence  of 
reading  problems  among  personnel  and  the  detrimental  effects  these  prob- 
lems were  having  on  the  conduct  of  Air  Force  OJT. 

Because  the  Air  Force  has  long  recognized  the  Impact  of  literacy 
on  training,  job  performance,  and  operating  costs,  it  has  funded  numer- 
ous R&D  efforts  in  this  area.  Burkett  (1976)  provides  an  excellent  re- 
view of  past  literacy  R&D  activities  in  the  Air  Force. 


AIR  FORCE  APPROACH  TO  LITERACY 

The  Air  Force  has  defined  its  literacy  problem  in  terms  of  the  dis- 
crepancy or  "gap"  between  the  reading  demands  of  training  and  job  materi- 
als, and  the  reading  skills  possessed  by  the  personnel  who  use  those 
written  materials  (Mockovak,  1974) . The  general  research  thrust  of  the 
Air  Force  to  reduce  this  discrepancy  has  been  a two-pronged  approach  — 
one  dealing  with  the  simplification/modification  of  materials  to  reduce 
the  reading  demands  of  the  printed  matter,  and  the  other  to  implement 
training  programs  aimed  at  increasing  the  literacy  skills  of  the  individ- 
ual (Burkett,  1976). 
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Reducing  the  Difficulty  Level  of  the  Printed  Materials 

Reducing  Che  difficulty  level  of  Che  printed  materials  is  an  impor- 
tant approach  in  closing  the  literacy  gap.  However,  this  approach  is 
limited  to  the  extent  to  which  the  materials  can  be  simplified  before  a 
degradation  in  the  accuracy  and  completeness  of  technical  information 
occurs.  Further,  this  approach  loses  its  appeal  at  the  point  where  it 
becomes  necessary  to  delete  the  more  demanding  job  knowledge  require- 
ments to  close  the  gap.  The  result  can  be  to  render  a person  useable 
only  in  a very  limited  job  assignment.  To  have  flexibility,  a person 
must  have  the  fundamental  literacy/cognitive  skills  to  adapt  to  new  job 
demands  as  the  situation  may  require. 

Increasing  the  Reading  Skills  of  Personnel 

Currently,  the  Air  Force  provides  reading  improvement  instruction 
to  airmen  scoring  below  certain  screening  test  criteria  at  two  different 
points  in  the  training  process:  during  basic  training,  and  following 
duty  assignment  to  the  field.  The  programs  at  these  two  stages  of  train- 
ing are  independent  of  one  another.  The  following  paragraphs  briefly 
discuss  the  essential  characteristics  of  each  program. 


During  Basic  Training 

At  the  time  of  this  research,  all  Air  Force  enlistees  are  assigned 
to  Lackland  AFB,  Texas  for  basic  military  training  (BMT) . A 15-minute 
reading  test  (designated  RJS-1)  is  administered  to  all  recruits.  Non 
prior-service  airmen  who  score  below  the  sixth  reading  grade  level  (RGL) 
on  the  RJS-1,  and  all  Mental  Category  IV  personnel,  regardless  of  their 
RJS-1  score,  are  given  the  California  Achievement  Test  (CAT)  reading  sub- 
section. Those  airmen  who  score  below  the  sixth  RGL  on  the  CAT  are  then 
assigned  to  the  Reading  Proficiency  Unit.  While  assigned  to  the  reading 
proficiency  training  program,  trainees  receive  four  hours  of  reading  pro- 
ficiency training  and  four  hours  of  military  training  per  day. 

The  first  week  is  spent  in  a conventional  classroom  setting,  working 
on  word  power  and  phonics.  At  the  end  of  the  week,  a diagnostic  test  is 
administered  and  the  results  are  used  to  guide  the  person's  progress 
through  the  remaining  weeks  of  the  program.  This  latter  portion  of  the 
course  is  self-paced  and  relies  primarily  on  the  SRA  (Science  Research 
Associates)  Reading  Series  materials,  although  other  materials  are  avail- 
able. Once  assigned  to  this  unit,  students  receive  reading  instruction 
for  a period  of  up  to  eight  weeks.  Throughout  the  training  period,  the 
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student  can  become  eligible  for  early  release  from  the  reading  program 
by  progressing  through  the  seventh  grade  level  materials.  At  this 
point  he  is  administered  an  alternate  form  of  the  CAT.  If  he  demon- 
strates a sixth  grade  reading  level  on  the  CAT,  he  leaves  the  Reading 
Proficiency  Unit  and  joins  a basic  training  flight.  If  he  does  ^iot 
achieve  that  criterion  level,  the  airman  remains  in  the  proficiency 
unit  and  receives  additional  training.  At  the  end  of  eight  weeks,  two 
courses  of  action  are  open  for  those  who  do  not  achieve  the  sixth  grade 
level.  Either  they  are  discharged  from  the  Air  Force,  or,  if  the  situa- 
tion warrants  and  the  unit  commander  approves,  they  are  maintained  in  the 
unit  for  two  additional  weeks. 

The  reading  proficiency  training  program  at  Lackland  AFB  is  admin- 
istered, controlled,  and  funded  by  Air  Training  Command  (ATC)  through 
its  operating  budget.  The  reading  instruction  is  oriented  toward  the 
Improvement  of  the  airman's  general  reading  skills. 


Base  Level  Reading  Training 

Upon  completion  of  basic  training,  the  airman  may  be  sent  to  a 
resident  technical  school  or  straight  to  a directed  duty  assignment  (DBA). 
With  an  operational  assignment  to  the  field,  the  airman  enters  the  OJT 
system  for  upgrade  training  (UGT)  to  a fully  qualified  skill  level  in  the 
job  specialty.  The  Air  Force  dual-channel  OJT  system  provides  training 
for  enlisted  personnel  to  qualify  in  both  the  job  knowledge  and  job  pro- 
ficiency required  to  perform  duty  in  an  Air  Force  specialty  (AFS).  Air- 
men are  expected  to  Increase  their  job  knowledge  primarily  through  a pro- 
gram of  self-study  (correspondence)  of  Career  Development  Courses  (CDC) 
while  acquiring  job  proficiency  and  experience  by  performing  job  tasks 
under  supervision.  This  system  requires  that  the  airman  be  able  to  read 
and  comprehend  a large  volume  of  training  and  job  material. 

The  unit  OJT  administrator  schedules  all  upgrade  trainees  who  do  not 
have  a score  of  60  or  higher  on  the  "general"  aptitude  scale  of  the  Armed 
Service  Vocational  Aptitude  Battery  (ASVAB)  or  on  the  Airman  Qualifying 
Examination  (AQE)  to  take  a reading  test  (normally  the  USAFI  Achieve- 
ment Test  III).  The  airman  is  then  tested  for  word  knowledge  and  reading 
comprehension.  Those  who  score  less  than  the  ninth  grade  reading  level 
in  either  area  are  enrolled  .in  a base  reading  improvement  course  concur- 
rently with  the  job  assignment  and  study  of  UGT  materials.  In  addition, 
if  an  airman  has  demonstrated  an  Inability  to  progress  satisfactorily  in 
UGT,  he  may  be  referred  to  the  Base  Education  Office  for  testing.  If  the 
airmen  score  less  than  ninth  grade  level  on  word  knowledge  or  reading  com- 
prehension they  are  enrolled  in  the  local  Base  Education  Office  reading 
improvement  course. 


The  reading  Improvement  programs  conducted  for  personnel  at  the 
base  level  are  the  responsibility  of  the  Base  Education  Office.  An 
Air  Force  survey  of  these  programs  has  been  reported  by  Mockovak  (1974) . 
Ninety  percent  of  the  84  responding  bases  had  reading  Improvement  pro- 
grams with  a combined  enrollment  of  5,774  airmen  during  the  period  of 
* April  1972  to  1 April  1973.  Entry  Into  a program  was  generally  (83X  of 

the  bases)  based  on  the  criterion  of  an  RGL  of  less  than  ninth  grade,  as 
specified  In  AFM  50-23;  however,  there  was  a wide  variety  of  reading 
tests  used  for  screening  and  evaluation  within  these  programs. 

Criteria  for  successful  completion  of  the  program  were  frequently 
not  made  explicit  and  appeared  to  vary  considerably  from  base  to  base, 
although  most  expected  the  airman  to  achieve  the  ninth  RGL  by  the  end 
of  the  program.  Time  in  these  programs  ranged  from  an  estimated  24  class- 
room hours  to  240  hours,  with  a mean  of  76  hours. 

The  most  common  types  of  reading  problems  cited  (55X)  were  the  stu- 
dents' Inability  to  read,  comprehend,  and  pass  their  CDC  material;  prob- 
lems cited  less  frequently  were  a lack  of  basic  reading  skills  (28Z)  and 
English  as  a second  language  (11X) . There  was  tremendous  variation  in 
the  educational  background  of  students  from  base  to  base,  with  the  pro- 
portion of  non  high  school  graduates  ranging  from  5 to  100  percent  of 
the  student  enrollment.  Overall,  non  high  school  graduates  averaged  50 
percent  of  the  student  enrollment.  The  majority  of  the  students  were 
apparently  In  their  Initial  job  assignment  (62X  training  for  the  3-Skill 
Level);  and  another  25  percent  were  training  for  the  5-Skill  Level. 

These  students  came  primarily  from  ten  career  fields:  24. 9X  from  Air- 
craft Maintenance  (42  & 43),  11. 9X  from  Civil  Engineering  (54  & 55), 

11. 2X  from  Transportation  (60),  11. OX  from  Food  and  Fuel  Services  (62  & 

63),  12. 7X  from  Administration  (70),  8.3X  from  Supply  (64),  and  3.9X 
from  Security  Police  (81) . 

The  majority  of  the  base  reading  Improvement  programs  were  devel- 
oped and  taught  by  local  colleges  (43X)  and  high  schools  (25X).  For  the 
remainder,  28  percent  were  arranged  and  taught  by  independent  contract 
personnel,  and  only  four  percent  by  Air  Force  personnel.  Presumably, 
the  extensive  use  of  local  colleges  and  high  schools  was  related  to  the 
funding  arrangements  for  many  of  these  programs.  Almost  one-half  (49X) 
of  the  programs  were  funded  by  the  Veterans  Administration,  and  13  per- 
cent were  paid  for  by  the  local  school  districts  using  state  and  federal 
funds.  The  remaining  38  percent  were  financed  by  the  Base  Education  Of- 
fices.* Again,  like  the  Reading  Proficiency  Unit  at  Lackland,  the  emphasis 
In  these  base  level  reading  courses  Is  on  the  Improvement  of  the  airman's 
general  reading  skills,  rather  than  the  development  of  job-related  read- 
ing skills.  Table  1 summarizes  the  essential  characteristics  of  the  two 
current  Air  Force  Literacy  Training  Programs. 

*Since  early  1977,  cancellation  of  the  VA  PREP  Program  has  caused  the 
Air  Force  to  fund  all  these  programs  under  Program  IDEA. 
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TABLE  1.  ESSENTIAL  FEATURES  OF  CURRENT  READING 
IMPROVEMENT  PROGRAMS  IN  THE  AIR  FORCE 


r 


1 

, 

? 

: 
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FEATURES 

BASIC  TRAINING 

PERMANENT  DUTY  STATUS 

Site  of 
Training 

Lackland  AFB 

Permanent  Duty  Station,  Base 

Education  Office 

Enrollment 

Criteria 

Mandatory  enrollment 
for  all  Mental  Cate- 
gory IV  airmen  and 
others  scoring  below 
6.0  RGL  on  the  CAT. 

1.  Mandatory  enrollment  for  all 
airmen  with  a general  aptitude 
score  of  60  or  below  who  score 
below  9.0  RGL  or>  either  part  1 

or  2 of  USAFI-III  or  comparable  test. 

2.  Mandatory  enrollment  for 
airmen  Identified  as  having 
reading  difficulties  In  UGT  and 
who  score  below  9.0  RGL  on 
either  part  1 or  2 of  USAFI-III 
test. 

Training 

Status  of 
Attendee 

In  basic  military 
training. 

In  skill  upgrade  training  (UGT) 
to  3 or  5 Skill  Level. 

Length  of 
Training 

Variable  - up  to 

200  hours.  Trainee 
leaves  reading  train- 
ing when  he  attains 

6.0  or  higher  RGL. 

Variable  - from  24  hours  to  240 
hours  depending  upon  course 
length  established  by  Base 

Education  Office. 

Program 

Evaluation 

California  Achieve- 
ment Test  Is  used  to 
assess  attainment  of 
sixth  grade  reading 
level.  TdSt  Is  admin- 
istered when  the  stu- 
dent completes  all 

7th  grade  material  or 
the  end  of  8 weeks, 
whichever  comes  first. 

Variable.  Most  bases  use  at- 
tainment of  9.0  RGL  as  measured 
by  USAFI  Achievement  Test  III 
or  equivalent.  Others  focus 
on  such  measures  of  success  as 

Increased  student  motivation. 

Increased  reading  rate.  Improved 
vocabularies,  passing  of  H.S.  GED, 
and  test/retest  Increment  gains. 

Measurement  Instruments  vary  from 
base  to  base. 

A 
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TABLE  1.  ESSENTIAL  FEATURES  OF  CURRENT  READING  IMPROVE- 


MENT  PROGRAMS  IN  THE  AIR  FORCE  (Continued). 

FEATURES 

BASIC  TRAINING 

PERMANENT  DUTY  STATUS 

Program 

Student  is  assigned 

Students  are  enrolled  in  avail- 

Delivery 

to  reading  profi- 

able improvement  courses  at 

System 

ciency  unit  for  up 

Base  Education  Office.  Training 

, 

to  10  weeks. 

is  concurrent  with  UGT.  The 
airman  is  released  from  duty  to 
attend  course,  but  is  assigned 
to  a job  position  against  the 
unit  manning  documents. 

4 

Program 

General  reading  skill 

General  reading  skill  improve- 

;i 

Emphasis 

improvement  to  the 

6.0  RGL.  Primarily  a 

ment  to  the  9.0  RGL.  Variable 
program  from  base  to  base  at 

self-paced  program 
with  intermittent 
criterion  checks. 

the  discretion  of  instructor 
and  Base  Education  Office. 

•• 

Objective 

Trainee  to  attain  a 
reading  ability  of 

6.0  RGL. 

Varies  from  base  to  base,  with 

80%  of  bases  specifying  the  9.0 

RGL  attainment  by  attendees. 

Administration 

HQ  Air  Training  Com- 

Base Education  Office  arranges 

of  Program 

mand  (ATC)  active- 
duty  AF  personnel 

for  conduct  of  reading  improve- 
ment courses.  The  following 

make  up  instructional 
and  administrative 
staff. 

percentages  show  those  taught 
by  local  colleges  (43%);  local 
high  schools  (25%);  independent 
contract  instructors  (28%);  and  AF 
personnel  (4%). 

i 

Instructional 

Primarily  uses  the 

Specified  by  the  agency  conduct- 

Materials 

SRA  Better  Reading 

ing  the  training.  Course  con- 

•: 

Books,  Reading  Labor- 
atory & Pilot  Labora- 
tory Series,  & Read- 
er's Digest  skill 

tent  varies  from  base  to  base 
at  the  discretion  of  the  in- 
structor teaching  the  course. 

level  builders.  Consists  of  self-contained,  programmed 

workbook  material,  with  multiple-choice  comprehension 

tests  over  previous  major  segments.  Graded  difficulty 
levels. 

-■•".jW  — . - -A  ^ - . A.~  
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TABLE  1.  ESSENTIAL  FEATURES  OF  CURRENT  READING  IMPROVE- 
MENT PROGRAMS  IN  THE  AIR  FORCE  (Continued). 


FEATURES 

BASIC  TRAINING 

PERMANENT  DUTY  STATUS 

Funding 

Source(s) 

ATC  operating  funds. 

Financed  through  a variety  of 
sources:  Veterans  Admlnlstra- 

tlon  (VA)  PREP  (49%)*  Base  Edu- 
cation Office  (38%),  Local 

School  District  (13%). 

Conclusions  Regarding  Current 

Air  Force  Reading  Improvement  Instruction 

Drawing  on  the  Information  obtained  In  his  survey,  Mockovak  (1974) 
concluded  that: 

1.  There  were  significant  numbers  of  Air  Force  personnel 
who  needed  and  were  enrolled  In  reading  Improvement 
programs  at  their  permanent  duty  stations. 

2.  The  Air  Force  lacked  a systematic,  standardized,  systems- 
orlented  approach  for  dealing  with  reading  training  prob- 
lems. 

3.  Each  base  had  Its  own  program,  resulting  in  a "myriad 
of  approaches,  varying  course  lengths,  different  defi- 
nitions of  successful  student  performance,  diverse  fi- 
nancing and  teaching  methods,  and  inadequate  records 
concerning  student  problems,  personnel  data,  and  progress." 

In  addition,  Mockovak  questioned  the  extent  to  which  Improvement  In 
job-related  reading  could  be  expected  from  programs  geared  to  develop 
reading  skills  In  the  context  of  general  educational  development.  Infor- 
mation obtained  in  the  survey  indicated  that  the  most  common  complaint 
centered  on  the  difficulty  individuals  had  In  comprehending  and  success- 
fully completing  CDC  materials.  Individuals  enrolled  In  reading  improve- 
ment programs  tended  to  have  reading  skills  averaging  slightly  below  the 
9th  grade  level.  Thus,  even  if  the  General  Educational  Development  (GED) 
programs  raised  reading  skills  to  the  9th  grade  level  (the  objective  for 
present  Air  Force  redding  programs),  the  student  would  still  be  faced  with 
the  problem  of  having  to  learn  the  specific  vocabulary  and  concepts  con- 
tained in  CDC  materials  before  being  able  to  perform  at  a new,  improved, 
general  reading  level. 

The  above- noted  limitations  of  the  Air  Force's  current  literacy 
training  programs  highlighted  the  need  for  Air  Force  development  of  a 
job-related  reading  program  as  opposed  to  the  general  reading  programs 
currently  available  at  Air  Force  bases.  Such  a program  would  be  different 
in  two  distinctive  ways  from  a general  reading  training  approach  — name- 
ly, the  f ocus  of  the  training  on  job  reading  tasks,  and  the  use  of  Air 
Force  training  and  job  materials  as  the  resource  bases  for  developing  read- 
ing Improvement  course  content. 

Much  behavioral  science  research  has  indicated  that  learning  is 
more  likely  to  transfer  from  the  school  to  the  job  situation  when  the 
school  tasks  closely  resemble  the  Job  tasks.  In  the  context  of  reading 
training,  this  requires  an  identification  of  Job  literature  and  an  anal- 
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ysis  of  its  reading  task  demands;  thus,  job  reading  improvement 
training  should  emphasize  extraction  of  job-related  information  from 
print  when  the  information  is  presented  in  the  special  formats  and 
organizational  styles  characteristic  of  printed  materials  used  on  the 
job. 

A job-related  approach  also  requires  the  use  of  actual  job  and 
training  printed  matter  as  the  resource  base  upon  which  reading  im- 
provement training  activities  are  based.  It  is  from  an  analysis  of 
job-specific  literature,  such  as  Air  Force  manuals,  regulations,  tech- 
nical orders,  pamphlets,  career  development  course  (CDC)  materials,  and 
specialty  training  standard  (STS)  study  references  that  the  specific 
technical  vocabulary  and  reading  tasks  are  delineated. 

OBJECTIVE  OF  THE  PRESENT  EFFORT 

In  response  to  the  induction  of  marginally  literate  personnel 
under  Project  100,000,  Apd  the  recognition  of  the  limited  success  of 
previous  literacy  program's.  Department  of  Defense  sponsored  a series 
of  research  projects  to: 

1.  Study  and  develop  methodologies  for  determining 
functional  literacy  levels  of  military  jobs 
within  the  Army. 

2.  Determine  functional  literacy  levels  for  six 
major  military  occupational  specialties  (M0S) 
into  which  large  numbers  of  marginally  literate 
persons  are  apt  to  be  assigned. 

3.  Develop  a prototype  literacy  training  program 
designed  to  provide  a level  of  functional  literacy 
appropriate  to  present  minimal  M0S  reading  requirements. 

In  view  of  this  research  (Sticht,  1975)  the  Air  Force  sponsored 
an  effort  to  adapt  and  apply  a job-related  literacy  approach  to  Air 
Force  literacy  training  needs. 

The  purpose  of  the  present  effort  was  to  develop  and  implement 
a prototype  J0RP  which  stressed  the  acquisition  and  development  of 
job-related  reading  skills  for  Air  Force  personnel.  Its  major  objec- 
tives were: 

1.  To  demonstrate  the  feasibility  of  using  a job-related 
approach  to  reading  instruction  with  airmen  in  the 
Air  Force  training  system. 
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2.  To  test  Che  effectiveness  of  this  approach  in  an  oper- 
ational setting  for  improving  airmen's  performance  in 
using  job-related  reading  materials. 

The  work  effort  was  accomplished  in  two  major  phases;  the  first 
being  the  design  and  development  of  the  instructional  materials  (JORP) , 
and  the  second  being  the  field  test  and  evaluation  of  the  instructional 
effectiveness  of  the  JORP  curriculum  materials. 


Chapter  2 

DEVELOPMENT  OF  THE  JORP 


DIFFERENCES  BETWEEN  JORP  AND  FLIT 

As  mentioned  earlier,  the  Army  sponsored  the  development  of  a 
functional  literacy  program  (FLIT)  to  increase  individuals'  functional 
literacy  to  a level  appropriate  to  minimal  MOF  (Military  Occupation 
Specialty)  reading  requirements.  The  present  project  was  intended 
to  adapt  the  FLIT  methodology  and  procedures  to  personnel  experiencing 
reading  difficulties  in  the  Air  Force.  The  following  paragraphs  point 
out  the  major  differences  from  FLIT  and  the  constraints  that  were  oper- 
ating during  the  development  of  JORP  that  influenced  the  design  of  the 
instructional  materials.  Refer  to  Sticht  (1975)  for  c thorough  descrip- 
tion of  curriculum  development  for  FLIT.  The  Army  is  operating  the  Ad- 
vanced Infantry  Training  Preparatory  Training  (AITPT)  School  as  the  cur- 
rent implementation  of  the  FLIT  model  developed  by  HumRRO. 


JORP  Design  Differences  Based  on  FLIT  Developmental  Exper  . 

A major  feature  of  FLIT  was  the  use  of  actual  Army  technical  publi- 
cations as  the  resource  base  for  developing  instructional  materials. 

While  this  characteristic  is  a desirable  instructional  feature  for  job- 
related  reading  training,  it  did  present  some  operational  problems  in 
the  administration  of  the  school.  First,  there  was  considerable  diffi- 
culty concerning  the  Army  publication  distribution  system's  capability 
to  service  the  large  volume  requirements  of  an  operating  school.  A sec- 
ond, and  more  serious  problem  arose  as  the  information  in  the  technical 
publications  was  superseded,  rescinded,  added  to,  or  otherwise  changed. 

A single  change  in  a manual  necessitated  multiple  changes  to  keep  the 
FLIT  materials  current. 

In  order  to  overcome  these  difficulties  while  preserving  the  use  of 
actual  job  information  as  a resource  base  for  JORP,  it  was  decided  to 
excerpt  pages  from  Air  Force  technical  publications  upon  which  reading 
instruction  activities  could  be  based.  These  excerpted  pages  were  then 
bound  together  in  a single  document  and  made  a reference  source  for  stu- 
dents to  use  in  completing  their  reading  training  activities. 

During  the  FLIT  implementation  in  the  Army,  it  was  feasible  to  inte- 
grate the  job-reading  training  and  job  skills  training  in  a technical 
training  school  environment  (Sticht,  1975).  While  this  necessitated 
some  changes  in  the  way  the  FLIT  program  operated,  student  performance  on 
the  Job  Reading  task  tests  was  approximately  equivalent  to  that  under  other 
delivery  systems,  even  though  the  time  available  for  job-reading  training 
was  shortened.  A concern  of  the  Air  Force  was  that  the  job-reading  program 
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be  Integrated  in  the  present  training  system  with  as  little 
disruption  as  possible.  This  concern  meant  that  the  course  could 
not  be  implemented  at  technical  school,  and  that  the  time  available 
for  the  JORP  reading  improvement  effort  would  be  limited,  since  the 
student  would  have  to  be  released  from  his  regular  duty  assignment 
to  attend  the  job-reading  class. 

Because  of  this  time  constraint  in  the  Air  Force,  the  FLIT  pro- 
gram was  adapted  in  several  ways  to  suit  Air  Force  needs.  Two  in- 
structional modules  were  deleted  from  the  six  modules  in  the  FLIT 
program.  These  were  the  modules  dealing  with  skill  practice  in  the 
job-reading  tasks  of  using  a table  of  contents  and  an  index.  It  was 
felt  the  students  coming  to  the  Air  Force  program  would  probably  ben- 
efit least  from  practice  in  these  modules,  as  their  expected  entry 
skills  would  probably  be  adequate  to  meet  these  demands  of  the  job. 
Additionally,  the  languaglng  activities  of  FLIT  were  deleted,  because 
it  was  decided  that  conceptualizing  activities  would  be  more  bene- 
ficial to  the  Air  Force  students. 


JORP  Design  Contraints  Based  on 

Air  Force  Requirements  and  Characteristics 

Several  additional  adaptations  of  the  FLIT  design  were  necessi- 
tated by  the  following  requirements  and  characteristics  of  the  Air 
Force. 

1.  The  reading  training  goal  of  FLIT  was  a 7.0  RGL,  while 

the  objective  of  JORP  was  to  be  a 9.0  RGL,  as  specified  in  AFM  50-23. 

2.  The  student  input  RGL  was  below  6.0  for  FLIT  and  expected 
to  be  between  the  6.0  and  8.9  RGL  for  JORP.  In  addition.  Air  Force 
students  who  were  unsuccessful  in  completing  career  correspondence 
training  because  of  reading  difficulties  were  to  be  eligible  for  the 
JORP  training. 

3.  The  FLIT  training  was  provided  prior  to  assignment  to  a 
technical  training  course,  whereas  in  the  JORP,  the  traininf  was  to 
be  Integrated  in  the  duty  day  of  permanent  party  personnel  already 
on  the  job. 

4.  The  time  available  for  reading  training  in  FLIT  was  6% 
hours  per  day  for  five  days  a week  for  six  weeks  (195  hours),  while 
the  maximum  JORP  training  time  was  that  available  under  the  ongoing 
reading  improvement  program  at  the  selected  field  evaluation  site. 
This  time  was  later  determined  to  be  2*s  hours  per  day  for  five  days 
a week  for  six  weeks  (75  hours). 
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SELECTION  CRITERIA  FOR  JORP  CLUSTERS 

The  scope  of  the  work  effort  was  limited  to  the  development  of 
a prototype  job  reading  program  that  could  be  applied  to  two  broad 
classes  of  Air  Force  job  career  clusters,  namely  maintenance  and  non- 
maintenance areas,  In  selecting  suitable  career  fields  for  Inclusion 
In  each  cluster,  the  following  factors  were  considered. 

1.  Two  candidate  career  fields  should  come  from  a maintenance 
job  cluster;  l.e.,  from  jobs  dealing  primarily  with  maintenance  func- 
tions (inspecting,  repairing,  servicing,  troubleshooting,  and  replac- 
ing) of  "hard"  aircraft  and  aircraft  equipment.  These  career  fields 
should  make  extensive  use  of  Air  Force  technical  orders. 

2.  Two  candidate  career  fields  should  come  from  a non-maintenance 
job  cluster;  l.e.,  from  jobs  encompassing  administrative  functions  (pre- 
paring, controlling,  distributing,  and  maintaining  documents)  associated 
with  many  types  of  paperwork  and  "soft"  systems  procedures  in  the  Air 
Force. 

3.  The  candidate  career  fields  should  have  a large  number  of  job 
Incumbents,  and  be  well  represented  at  almost  any  Air  Force  base  (AFB) . 
This  was  necessary  so  that  a sufficient  sample  of  subjects  could  be 
drawn  for  the  tryout  and  field  test  of  the  JORP  at  one  AFB.  Also,  the 
larger  the  career  fields  in  absolute  numbers,  the  better,  since  the  pro- 
totype would  apply  directly  to  a greater  number  of  people. 

4.  The  candidate  career  fields  should  have  sufficient  numbers  of 
low  ability  readers  who  have  been  identified,  or  could  be  identified,  as 
having  reading  difficulties  with  Air  Force  job  and  upgrade  training  read- 
ing tasks,  so  that  a sufficient  sample  size  could  be  obtained  during  the 
field  test. 

5.  The  candidate  career  fields  should  have  a sufficient  "literacy 
gap"  (l.e.,  the  discrepancy  between  the  Job  reading  skills  of  job  incum- 
bents and  the  reading  requirements  of  the  career  field).  It  would  be  of 
little  utility  to  develop  a JORP  for  a career  field  wherein  no  need  ex- 
isted because  the  job  Incumbents  were  adequately  accomplishing  the  read- 
ing tasks. 

The  above  selection  criteria  required  data  concerning  the  number  of 
airmen  assigned  in  the  different  Air  Force  specialty  codes  (AFSC),  the 
extent  of  the  literacy  gaps  In  Air  Force  career  fields,  and  the  number 
of  airmen  who  have  been  identified  as  having  reading  difficulties  by 
career  field.  The  most  recent  and  relevant  data  pertaining  to  this  area 
is  that  reported  in  Mockovak's  series  of  reports  (Mockovak,  1974,  1974a, 
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1974b).  These  data  provided  the  basis  for  selection  of  the  source 
materials  to  be  Included  In  each  prototype  JORP  cluster. 

For  the  non-maintenance  JORP  cluster,  three  AFSCs  were  selected: 

702X0  - Administration  Specialist,  645X0  - Inventory  Management  Spe- 
cialist, and  645X0  - Materiel  Facilities  Specialist.  For  the  mainte- 
nance cluster,  five  AFSCs  were  selected:  421X2  - Aircraft  Pneudralic 
Systems  Repairman,  431X1A  - Aircraft  Maintenance  Specialist  (Recipro- 
cating Engine  Aircraft),  431X1C  - Aircraft  Maintenance  Specialist  (Jet 
Aircraft,  One  and  Two  Engine),  431X1F  - Aircraft  Maintenance  Specialist 
(Turboprop  Aircraft),  and  431X1E  - Aircraft  Maintenance  Specialist  (Jet 
Aircraft,  Over  Two  Engines). 

DISTINCTIONS  BETWEEN  INSTRUCTIONAL  STRANDS 

As  in  the  FLIT  program,  the  JORP  materials  were  developed  in  two 
major  strands.  Although  both  instructional  strands  of  the  JORP  trained 
airmen  to  locate,  analyze,  and  comprehend  job-related  information,  the 
two  strands  used  different  approaches.  In  the  Strand  I portion  of  the 
program,  students  use  their  existing  literacy  skills  to  practice  speci- 
fic job-reading  tasks.  In  Strand  II,  they  developed  and  improved  basic 
reading  and  thinking  skills.  In  addition.  Strand  II  directly  presented 
job  concepts  and  vocabulary.  Other  distinctions  between  the  strands  are 
summarized  in  Table  2. 

Inasmuch  as  Strand  II  demands  more  basic  reading  skills,  it  appears 
to  be  preparation  for  Strand  I.  Actually,  the  two  strands  are  taught 
side-by-side  throughout  the  program,  thus  reinforcing  and  complementing 
each  other.  Strand  I exercises  enable  students  to  practice  basic  reading 
skills  on  specific  job-reading  tasks;  Strand  II  exercises  increase  the 
students'  knowledge  of  job  concepts  and  vocabulary,  while  strengthening 
the  base  on  which  the  job  reading  skills  rest.  A description  of  the  char- 
acteristics of  each  instructional  strand,  and  the  development  procedures 
employed  in  each  strand  is  provided  in  the  following  paragraphs. 
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TABLE  2.  SUMMARY  OF  DISTINCTIONS  BETWEEN  STRAND  I & STRAND  II. 


STRAND  I 

STRAND  II 

STUDY 

Emphasizes  Individual 
practice,  with  very 
little  direct 

Instruction. 

Uses  group  activities  primarily, 
with  direct  Instruction  from  the 
teacher,  as  well  as  group  dis- 
cussion and  feedback.  It  Is  not 
self-paced. 

STUDENTS' 

RESPONSES 

Nearly  all  written. 
(Answers  on  worksheets 
and  tests.) 

Primarily  oral  and  graphic. 

(Group  discussions  of  student's 
charts,  drawings,  and  tables.) 

SOURCE 

MATERIAL 

Excerpted  directly  from 
job  and  training  man- 
uals. 

A series  of  passages  written 
specifically  for  the  JORP,  each 
one  dealing  with  a different  job 
content  area.  Moreover,  these 
passages  are  deliberately  writ- 
ten at  a lower  level  of  diffi- 
culty than  typical  job  reading 
materials. 

PRESENTATION 
OF  CONCEPTS 
& VOCABULARY 

Presented  to  the  extent 
of  occurrence  in  the 
extracts  for  technical 
orders,  manuals,  etc. 

Makes  a more  deliberate  presen- 
tation via  the  content  of  the 

Job  Reading  Passages  and  the 
conceptualizing  activities 
required  of  the  students. 

STUDENT 
FEEDBACK  & 
EVALUATION 

Criterion-based  and 
objective.  (Answers  to 
questions  on  worksheets 
and  tests.) 

Subjective.  Primarily  the  re- 
sponses to  a student's  work  by 
his  peers  in  group  discussion. 
There  are  no  absolutely  right 
or  wrong  answers,  rather,  vary- 
ing degrees  of  sophistication 
at  conceptual  activities. 

TESTS 

Timed,  as  reflected 

In  the  passing  criteria. 

There  Is  no  timed  work  In  this 
strand. 

CHARACTER 

OF 

ACTIVITIES 

Analytic.  This  strand 
requires  the  student  to 
break  the  content  of  what 
he  Is  reading  down  to 
bits  of  Information  small 
enough  to  serve  as  an- 
swers on  tests  and  work- 
sheets. 

Synthetic.  This  strand  requires 
the  student  to  put  together 
bits  of  Information  from  a 
whole  passage  or  paragraph  to 
make  a complete  chart,  table,  or 
drawing  on  a particular  subject. 
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Strand  I Development 

The  purpose  of  Strand  I is  to  improve  the  student's  skill  at  four 
reading  tasks  typically  encountered  in  job  training  and  on  the  job. 

This  approach  is  based  on  the  premise  that  the  student  comes  to  the  pro- 
gram already  equipped  with  such  basic  reading  skill  as  decoding.  The 
Reading  Proficiency  Unit  at  Lackland  AFB  graduates  students  at  the  6th 
RGL,  so  the  JORP  was  aimed  at  airmen  at  the  6th  through  9th  RGL.  Strand 
I emphasizes  the  application  of  existing  literacy  skills  to  examples  of 
typical  Air  Force  job  reading  and  training  materials.  The  following  six 
instructional  principles  directed  the  development  of  Strand  I.  All  of 
these  have  been  used  with  success  in  other  educational  and  training  sit- 
uations, Including  project  FLIT. 

e Individualized  Instruction  — The  students  progress  at  their 
own  rate,  using  materials  from  their  own  career  cluster. 

# Performance-Oriented  Instruction  — The  students  perform  the 
kinds  of  reading  tasks  encountered  in  job  training  and  on  the 
job.  Thus,  there  should  be  a direct  transfer  of  the  skills 
learned  in  the  JORP  to  the  job  itself. 

# Functional  Instruction  — The  students,  ideally,  use  actual 
job-reading  material,  not  general  reading  materials,  and  there- 
by should  see  the  purpose  of  training  in  terms  of  job  profi- 
ciency. 

e Student-Assisted  Instruction  — The  students  may  participate 
as  administrative  aides  and  peer  instructors  to  relieve  pres- 
sure on  teachers  and  to  reinforce  what  they  have  just  learned. 

e Programmed  Instruction  — The  students  advance  through  lin- 
early programmed  modules  according  to  their  performance  on 
proficiency  tests.  Each  module  has  branching  loops  for  reme- 
dial instruction. 

# Quality-Controlled  Instruction  — The  students  should  develop 
mastery  of  the  reading  task  before  proceeding  to  the  next  module. 

To  put  these  principles  into  operation,  the  JORP  uses  the  following 
materials  in  the  classroom:  (1)  Proficiency  Tests,  (2)  Worksheets,  (3) 
a Job  Reading  Manual  of  source  material,  and  (4)  a Student  Control  Card. 
The  first  three  are  used  by  the  student  to  practice  job  reading  skills; 
the  fourth  is  used  by  the  teacher  to  keep  track  of  the  student's  progress 
and  determine  what  the  student  should  do  next.  In  developing  all  of  these 
materials,  three  guidelines  were  used. 
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1.  Strand  I should  concentrate  on  the  skills  of  extracting 
information,  leaving  to  Strand  II  the  practice  of  draw- 
ing inferences  and  conclusions  and  making  generalizations. 

2.  The  materials  should  apply  to  the  principles  and  pro- 
cedures that  Air  Force  workers  and  first-line  supervisors 
are  expected  to  know  and  use  for  their  jobs.  Information 
addressed  to  higher  levels  of  command  should  not  be  in- 
cluded. 

3.  The  readability  of  the  worksheets  should  be  near  the  9th 
reading  grade  level. 

The  student  in  Strand  I practices  each  of  the  job  reading  skills 
identified  below  in  a separate  module  of  instruction,  and  passes  from 
one  module  to  the  next  as  a result  of  performance  on  proficiency  test. 
Whenever  the  students  fail  a proficiency  test,  they  complete  a set  of 
worksheets. 

The  Narrative  module  provides  practice  at  extracting  information 
from  narrative  prose  in  a manual,  such  as  the  operating  principles  of 
an  aircraft  engine,  or  a list  of  safety  rules.  Figure  1 shows  a typi- 
cal page  from  a Job  Reading  Manual  narrative  section.  A typical  ques- 
tion asked  is:  "According  to  the  written  information  on  page  47,  where 
does  the  integral  brake  system  obtain  braking  power?".  For  an  example 
of  a complete  Narrative  worksheet,  see  Figure  2. 

The  Procedural  Directions  module  provides  practice  at  extracting 
information  from  procedural  directions,  like  the  instructions  for  com- 
pleting a pre-flight  check.  It  differs  from  the  Narrative  module  in 
two  respects.  First,  the  source  material  in  the  Job  Reading  Manual  for 
Procedural  Directions  always  presents  a sequence  of  steps  that  must  be 
done  in  a particular  order.  Even  though  some  Narrative  excerpts  deal 
with  operational  procedures  they  are  not  the  step-by-step  directions 
for  accomplishing  the  job. 

Second,  there  is  one  type  of  question  found  only  in  the  Procedural 
Directions  modules.  This  type  requires  the  student  to  rearrange  into 
correct  sequence  a series  of  operational  steps  that  are  presented  out 
of  order  on  the  worksheet.  Other  Procedural  Directions  questions  are 
similar  to  those  in  Narrative.  Examples  of  both  are  found  in  Figure  3. 
Figure  4 shows  the  corresponding  Job  Reading  Manual  page. 
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The  Schematics  module  differs  from  Narrative  only  in  the  type  of 
source  material  used  for  the  Job  Reading  Manual.  Tables,  graphs, 
charts,  maps,  drawings,  diagrams,  etc.,  are  referred  to  by  the  student 
to  answer  such  questions  as,  "Use  the  graph  in  Figure  5-11  to  tell  how 
much  you  will  tighten  the  LBGG  cable  when  the  temperature  is  80°  F." 
Note,  Incidentally,  that  this  question  deals  with  an  operational  pro- 
cedure (trimming  the  flight  control  cables),  but  that  it  is  not  appro- 
priate for  the  Procedural  Directions  module  because  there  is  no  se- 
quence involved.  Figures  5 and  6 illustrate  a Schematic  worksheet  and 
the  corresponding  Job  Reading  Manual  excerpt,  respectively. 
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/.  When  the  signal  is  given,  gradually  in- 
crease the  engine  power  setting  until  the  aircraft 
starts  to  move. 

8-4.  After  the  aircraft  starts  to  move,  bacV 
off  on  the  throttle  to  maintain  a low,  safe  taxi 
speed.  Reduce  this  speed  to  10  miles  per  hour 
in  congesttd  areas  and  to  even  lower  speed  in 
areas  that  have  loose  gravel,  ice,  snow,  or  sand, 
and  remember  to  slow  the  aircraft  down  belore 
starting  a turn. 

8-5.  la  taxiing,  the  aircraft  is  turned  with  the 
nosewheei  sterling  system  if  so  equipped.  To 
start  each  turn,  fust  make  only  a slight  change 
in  heading.  Then  gradually  increase  the  change 
so  heading  until  you  get  the  desired  rate  of  turn. 
Use  the  same  technique  coating  out  of  the  turn  in 
order  to  avoid  sudden  changes  of  direction  that 
put  unusual  stesses  on  the  nosewheei. 

8-6.  Improper  use  of  brakes  during  taxiing 
also  places  severe  stresses  on  the  aircraft.  To  use 
the  brakes  correctly,  first  depress  the  pedals  to 
obtain  a reasonable  rate  of  deceleration.  Then, 
as  the  aircraft  slows  down,  gradually  release  the 
brakes  so  that  you  are  applying  very  little  brak- 
ing pressure  when  the.  aircraft  stops. 

8-7.  Now  that  you  know  how  to  use  the 
brakes,  let's  point  out  a few  dco'ts  for  taxiing. 
Here  they  are: 

a.  Unless  you  are  on  an  established  taxiway, 
don't  taxi  an  aircraft  within  100  feet  of  an  active 
runway. 

.b.  Don't  taxi  within  10  feet  of  any  obstruc- 
tion. 

e.  Don't  taxi  at  night  with  the  aircraft  lights  oft 

d.  Don't  use  aircraft  lights  in  sneb  a way  as  to 
blind  ground  crcsvmeo. 

e.  Don't  open  aircraft  doors  or  hatches  during 
taxiing. 

/.  When  moving  from  a row  of  parked  aircraft, 
don't  use  excessive  engine  power.  (Use  the  least 
cmounl  of  power  tbat  produces  forward  move- 
ment) 

8-8.  So  far  in  our  discussion  of  taxiing,  we 
have  considered  the  work  done  by  the  man  in  the 
cockpit  Now  let’s  turn  our  attention  to  the  work 
done  by  the  ground  crew. 

8-9.  Ground  crew  dutits.  Before  an  aircraft  en- 
ters or  leaves  a ramp,  a taxi  signalman  is  required 
to  direct  its  roos-rniect  1 et's  see  what  he  does. 

8-10.  The  taxi  signalman  stands  in  front  of  the 
aircraft.  Hi;  position  should  be  sc  '.hat  he  is  svlthin 
sight  of  the  pilot  and  to  the  left  ol  the  aircraft.  In 
this  position  he  can  see  the  eyes  of  the  pilot  at 
all  times,  and  the  pilot  can  see  all  signals  that 
are  given  by  him. 

8-11.  The  taai  signalman  uses  signals  like  (hose 
used  for  towing.  Some  of  these  signals  have 
already  been  illustrated  and  discussed  in  the  text. 


For  a review  of  the  sirnals,  refer  to  figure  6 
and  to  AFR  60-1 1. 

8-12.  Next  let  us  consider  the  work  done  by 
members  of  the  ground  crew  other  than  th:  sig- 
nalman. 

8-13.  In  some  instances  during  taxiing,  ground 
crewmembers  may  act  as  wing  wallers  or  Gag- 
men. Wing  walkers  are  required  if  an  aircraft 
will  be  taxied  within  25  feet  of  aa  obstruction  or 
it  the  aircraft  to  be  taxied  is  not  a locally  based 
one.  In  addition,  after  the  landing  of  an  aircraft 
that  is  not  locally  based,  a Caemea  cr  a ‘‘follow 
me"  vehicle  usually  will  direct  the  craft  to  the  ap- 
propriate parking  area.  Oaca  the  aircraft  reaches 
the  parking  area,  however,  a maintenance  spe- 
cialist usually  takes  over  as  signalman  and  directs 
the  parking  of  the  aircraft. 

8-14.  Parking.  When  you  park  an  aircraft,  the 
procedures  that  you  use  depend  upon  whether 
you  are  parking  it  for  a few  hours  (temporary 
parking)  or  overnight  (extended  parking).  An- 
other determining  factor  is  whether  or  net  ex- 
treme weather  conditions  exist.  The  following  in- 
formation on  parking  is  limited,  primarily,  to  the 
C-123  aircraft. 

8-15.  Temporary  parking.  The  -2  TO  (or 
equivalent)  provides  you  with  gross  weight  versus 
wind  velocity  graphs  similar  to  (he  one  illus- 
trated in  figure  7.  (If  a graph  is  not  provided, 
wind  velocity  limits  are  given.)  The  grsoh  de- 
termines whether  you  can  park  the  aircraft  or 
must  moor  it  Notice  in  figure  7 tbat  as  loag 
as  the  gross  weight  versus  wind  (or  gust)  ve- 
locity values  are  within  area  A,  you  can  safely 
park  the  aircraft.  Otherwise,  ycu  nust  apply  one 
of  the  appropriate  mooring  tiedown  procedures. 
To  park  the  C-123,  observe  the  following  gen- 
eral procedures,  which  are  applicable  (with 
slight  variation)  to  most  aircraft: 

a.  Position  the  aircraft  to  provide  clearance 
for  maintenance,  servicing,  and  tire  lanes  and, 
whenever  possible,  with  at  least  a 20-foot  svieg- 
np  distance  from  other  airplanes. 

b.  Don't  park  the  airplane  closer  than  750 
feet  from  the  centerline  ol  landing  strips. 

c.  Don’t  park  tbe  aircraft  in  line  with  th:  cr.ds 
of  landing  ships  within  1,000  feet  cr  parking  strip 
ends. 

d.  Don't  park  tbe  airplane  closer  than  250 
feet  from  the  farthest  edge  of  connecting  taxi 
snips.  If  you  ever  find  it  recessary  to  tem- 
porarily park  an  aircraft  wiJt  any  psrt  of  it  ex- 
tending  into  a taxiway,  station  an  observer  to 
warn  oncomiog  traltic.  At  night  the  observer 
should  be  equipped  with  a suitable  light;  during 
the  day,  with  a suitable  flag. 

r.  Whenever  possible,  head  the  airplane  into 
the  wind. 


Figure  1.  Sample  Page  from  Job  Reading  Manual,  Narrative  Section 
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Page  24  contains  written  instructions  for  taxiing  and  parking  the 
aircraft.  You  will  be  better  able  to  perform  your  job  if  you  are  good 
at  using  instructions  presented  in  written  form  like  this.  Tell  how  you 
would  respond  in  the  following  job  situations. 

1.  You,  as  a taxi  signalman,  ara  helping  to  park  tha  aircraft.  Where 
do  you  stand? 


2.  Why  do  you  stand  in  this  particular  position? 


3.  Ycu  have  qualified  to  taxi  the  aircraft.  How  close  to  an  obstruction 
can  you  safely  taxi? 


4.  You  arc  leaving  a row  of  parked  aircraft.  How  much  engine  power 
do  you  use?  (Reference:  paragraph  8-7.) 


5.  You  have  finished  taxiing  the  aircraft,  and  you  are  deciding  whether 
to  park  it  or  moor  it.  What  two  things  do  you  need  to  know  to  make 
your  decision? 


Figure  2.  Example  of  Narrative  Worksheet 
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6.  You  are  forced  to  park  the  aircraft  temporarily  with  a wing  tip 

exterding  into  a taxiway.  What  precaution  do  you  observe?  (Refer  to  para 

8-iSd.) 


7.  You  are  taxiing  the  aircraft  through  a congested  area,  on  a dry  con- 
crete taxiway.  How  fast  can  you  safely  go?  (Reference:  paragraph  8-4.) 


8.  You  are  teaching  a trainee  the  proper  taxi  signals  to  use  when  he 
works  on  the  ground  crew.  Where  can  he  look  up  an  illustration  or 
more  Information  about  the  signals?  (Reference:  paragraph  8-11.) 


9.  You  are  the  chief  of  a ground  crew  about  to  taxi  the  aircraft.  Bow 
do  you  decide  whether  or  not  you  need  any  men  on  your  crew  to  be  wing 
walkers?  (Reference:  paragraph  8-13.) 


10.  While  you  are  taxiing  the  aircraft,  why  is  it  so  important  to  use 
the  brakes  properly?  (Reference:  paragraph  8-6.) 


Figure  2.  Example  of  Narrative  Worksheet  (continued) 
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In  this  job  situation  you  arc  an  administrative  specialist 
establishing  requirements  with  the  Publications  Distribution  Center 
to  meet  the  needs  of  your  PDO  customers.  Page  36  contains  procedures 
you  can  use  to  establish  requirements  for  publications  with  PCD. 

1-6  In  what  order  will  these  job  task*  be  done? 

Report  customer  requirements  to  PDC  on  AF  Fora  764. 

Prepare  AF  Fora  574  for  each  publication  required. 

Advise  customer  of  deadlines  for  submission  of  requirements. 

_____  Record  requirements  on  AF  Fora  574  as  they  are  received. 

_____  Consolidate  all  your  customers'  requirements. 

Distribute  A F Publications  Bulletin  (PB)  to  all 

customers. 

7.  What  four  steps  will  you  do  to  validate  customer  requirements? 


Figure  3.  Examples  of  Procedural  Directions  Questions 
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8.  What  trill  you  do  with  AF  Fora  764A  after  recording  a customer's 
requirement? 


9.  What  will  you  base  your  requireaents  on  for  an  ”M"  (erica  publication 
to  PCD? 


10.  What  will  you  do  with  each  AF  Fora  574  prepared  for  each  F-type 
publication  required  by  custooers? 


Figure  3.  Examples  of  Procedural  Directions  Questions  (Continued) 
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Publication*  Requirements.  To  establish 
requirements  for  publications  with  the  Air  Force 
Publications  Distribution  Center  (AFPDC),  you, 
the  PDO,  must  take  the  fol  lowing  basic  steps,  These 
basic  steps  apply  to  publications  with  the 
distribution  symbols  M,  B,  S,  and  F. 

Step  1.  Distribute  the  Air  Force  Publications 
Bulletin  (PB)  to  all  of  your  customers.  Advise  them 
at  the  same  time  of  any  deadline  or  other  special 
instructions  for  submitting  their  requirement  to 
you. 

Step  2.  Prepare  a separate  AF  Form  574, 
Distribution  Record,  for  each  publication 
announced  in  the  1’B  for  which  requirements  must 
be  established. 

Step  3.  Record  your  customers*  requirements 
individually  on  the  applicable  AF  Form  574  as  they 
arc  received.  After  recording  your  customers* 
requirements,  you  may  either  destroy  their  AF 
Form  764a  or  keep  them  on  file  to  substantiate  your 
own  records. 

Step  4.  Consolidate  all  your  customers' 
requirements,  addiui;  appropriate  quantities  for 
stock,  and  report  them  to  the  PDC  on  AF  Form 
764. 

Allowable  stock  quantities  to  be  added  to  your 
customer  requirements  may  be  found  in  Chapter  3 
of  AFM7-I. 

Validation.  After  your  customers'  requirements 
have  been  received,  it  is  your  responsibility  to 
insure  that: 

• The  publications  are  needed  by  that 
organization. 

• The  request  is  properly  filled  out  according  to 
AFM  7-2  and  signed. 

• The  distribution  symbol  of  the  publication 
applies. 

• Functional  statements  apply  to  that 
organization. 

Strict  distribution  (M,  B,  and  S).  Customers  must 
Submit  requirements  to  you  on  their  own  initiative. 
You  must  insure  that  your  customers  understand 


that  series  distribution  publications  are  amouncsd 
in  Section  It  of  the  PB.  Customers  will  submit 
requirements  to  you  on  AF  Form  7c-4a  for  each 
publication  they  want  ia  a particular  series,  showing 
the  appropriate  symbol  in  the  space  provided  on  the 
form. 

Customers  should  not  submit  requirements  for 
publications  with  M or  B distribution  symbols 
unices  their  organizations  are  authorized  to  recerrv 
these  publications.  Futheneore,  they  should  not 
resubmit  requirements  for  any  publication  for 
which  they  have  already  established  requirements. 

Unlike  your  customers,  you  cannot  establish 
requirements  with  the  PDC  by  individual 
regulations  in  a series.  You  must  establish 
requirements  for  the  entire  series. 

You  can  usually  base  your  requirements  for  a 
particular  series  (M,  B,  or  S)  on  the  sum  of  your 
customers'  requirements  for  the  “MOST 
WANTED"  AFR  in  the  series.  This,  however,  is 
not  always  true.  If  using  the  "MOST  WANTED" 
rule  results  in  your  receiving  an  excessive  number 
of  other  regulations  in  the  series,  you  must: 

• Reduce  your  series  requirements  as  low  as 
you  can  without  creating  a shortage  in  your 
initial  distribution  (ID)  requirements. 

• Requisition  the  extra  copies  you  need  to 
complete  ID  of  a publication  when  it  appears 
in  Section  11  of  the  PB.  Identify  the 
publication  on  your  requisition  by  adding  the 
suffix  (BP)  to  the  short  title. 

The  (BP)  represents  that  the  item  is  being 
processed.  Your  (BP)  requisition  will  be  held  on 
back  order  at  tils  PDC  and  filled  as  soon  as  stock  is 
received  from  the  printers. 

Functional  distribution  (F).  Functional. type 
publications  are  treated  the  same  as  series  type 
except  you  set  a specific  date  for  your  customers  to 
report  their  requirements.  The  AF  Form  574, 
prepared  for  each  of  these  publications,  is  placed  in 
suspense  until  you  receive  ycur  customers' 
requirements.  Once  you  receive  you  customers' 
tequirements,  you  follow  the  same  basic  steps  as  for 
scries  distribution  publications.  Be  sure  to  check 
AFM  7-1  for  the  stock  quantity  to  be  added. 


Figure  4 


Corresponding  Procedural  Directions  Job  Reading  Manual  Page 
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In  this  job  situation  you  are  rigging  the  aircraft  brake  control 
cables.  Figure  5-11  on  page  7 presents,  in  scheoatic  fora,  inforaation 
that  you  need.  Use  Figure  5-11  to  tell  how  you  would  do  tbe  following 
job  tasks. 


1.  The  temperature  today  is  100*F.  At  what  tension  will  you  rig  the 
LGB8  cable? 


2.  What  does  the  1CBB  cable  do? 


3.  You  are  handling  a cable  with  a Yellov-White-Brown  color  code.  What 
does  it  do? 


4.  You  find  an  LGBA  cable  is  tightened  to  120  pounds,  and  you  know  that 
the  temperature  is  50*F.  Do  you  tighten  or  loosen  it?  How  much? 


Which  cables  are  pulled  forward  when  the  pilot  applies  the  brakes? 


Which  cable  moves  aft  if  the  copilot  releases  the  brakes? 


7.  You  are  looking  at  the  cable  that  is  furthest  to  the  left  side  of 
the  aircraft.  What  color  code  is  it? 


Figure  5.  Example  of  Schematic  Worksheet 
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8.  You  need  to  tlgheao  tha  LCBB  cabla  from  tha  pilot  (left  side  of 
aircraft)  to  tha  quadrant  naar  tha  forward  main  gear  bulkhead. 

How  uny  turubuckles  are  In  that  cabla  that  you  could  usa  to  adjust 
tension? 


9.  If  tha  temperature  today  Is  twenty  below  zero,  how  tight  will  you 
rig  tha  cables? 


10.  What  does  tha  cabla  run  through  In  the  bulkhead  forward  of  tha 
forward  wain  gear  bulkhead? 
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Figure  6.  Corresponding  Schematic  Job  Reading  Manual  Pag< 
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The  Forms  module  provides  practice  at  following  one  kind  of  pro- 
cedural directions:  how  to  prepare  Air  Force  forms.  In  addition,  it 
familiarizes  the  airman  with  the  format  of  forms.  The  worksheets  in 
this  module  lead  the  student  step-by-step  through  the  written  direc- 
tions for  filling  out  four  different  Air  Force  (AF)  and  Air  Force  tech- 
nical order  (AFTO)  forma.  The  emphasis  is  on  close  reading  of  the  in- 
structions in  order  to  be  able  to  make  the  correct  entries  on  the  form 
when  presented  with  a hypothetical  job  situation. 

A typical  worksheet  was  developed  by  first  selecting  a page  from 
a CDC  or  job  manual  that  provided  practice  at  locating  and  understand- 
ing job-related  information.  Such  a page  would  ordinarily  deal  with 
only  one  tpolc,  such  as  how  to  jack  an  aircraft  for  weighing  or  how  to 
requisition  publications.  Questions  were  then  written  that  encouraged 
or  required  reading  in  different  parts  of  the  page,  but  which  discour- 
aged aimless  "skinning".  The  questions  emphasized  extracting  specific 
information  from  the  passage.  Questions  which  required  a high  degree 
of  Inference  from  the  passage  as  a whole  were  avoided. 


Strand  II  Development 

Strand  II  of  the  JORP  is  designed  to  improve  basic  reading  and 
thinking  skills  by  providing  instruction  in  reading  and  comprehending 
specially  prepared  job-related  printed  materials.  The  major  instruc- 
tional tool  used  to  achieve  this  goal  is  a series  of  exercises  that  re- 
quire the  student  to  make  representational  transformations  from  narra- 
tive descriptive  passages  that  present  important  job  concepts. 


Instructional  Philosophy  & Background 

Humans  have  developed  a unique  system  for  transferring  knowledges 
to  each  other  by  language.  Both  speaking  and  writing  are  processes 
for  representing  thoughts  in  external  displays,  which  people  can  de- 
code to  form  Internal  displays  called  conceptualization.  Besides  the 
linguistic  modes  of  spoken  and  written  displays,  there  are  other  meth- 
ods for  representing  conceptualizations  externally.  For  Instance,  we 
can  draw  pictures,  or  produce  gestures  or  bodily  postures.  Or  we  can 
externally  represent  thoughts  through  a combination  of  linguistic  and 
non-llngui8tic  representations,  such  as  figures,  graphs,  tables,  et 
cetera. 

Representational  modes  can  be  classified  into  three  categories: 
linguistic,  iconic,  and  schematic.  Linguistic  representations  include 
speech  and  writing;  iconic  representations  use  pictures  to  represent 


conceptualizations;  and  schematic  representations  include  classifi- 
cation tables  and  flow  charts,  which  contain  iconic  structural  fea- 
tures and  also  linguistic  signs  like  labels  and  short  phrases.  There 
are  other  modes,  of  course  (e.g.,  graphs,  in  the  schematic  category) 
but  the  JORP  deals  only  with  these  three  modes  of  representation. 

The  different  representational  modes  are  ways  to  display  informa- 
tion. When  information  is  presented  to  a person,  it  might  be  in  one 
of  these  three  categories  — linguistic,  iconic,  or  schematic.  Who- 
ever attends  to  the  information  display  — receiving  the  information 
as  input  — can,  in  turn,  pass  the  information  on  to  someone  else  — 
as  output.  For  example,  someone  might  read  a book  (linguistic  input) 
and  then  write  or  tell  someone  all  about  it  (linguistic  output). 

When  information  is  transferred,  however,  its  representational 
form  does  not  always  stay  the  same.  In  the  example  just  given,  the 
person  who  read  the  book  (linguistic  input)  might  have  chosen  to  rep- 
resent his  conceptualization  of  the  book  by  drawing  a picture  (iconic 
output).  Or,  information  presented  in  a table  (schematic  input)  might 
be  used  as  source  material  from  which  a narrative  is  written  (linguis- 
tic output),  which  would  represent  essentially  the  same  meaning  as  the 
table.  In  these  cases,  the  meaning  of  the  information  remains  the 
same,  but  the  form  in  which  it  is  displayed  changes.  A transformation 
has  occurred. 

It  Is  the  transformation  from  one  representational  form  to  another 
that  Is  at  the  instructional  heart  of  Strand  II.  The  Strand  II  material 
provides  input  representations  in  two  modes  of  linguistic  display  — 
spoken  Instruction  by  the  teacher  and  specially  prepared  passages,  pre- 
senting job  concepts  and  information.  The  student  is  then  required  to 
transform  the  linguistic  display  Into  an  iconic  display  (such  as  a pic- 
ture representing  some  portion  of  the  written  passage)  or  a schematic 
display  (such  as  a flow  chart  or  classification  table).  Having  made 
this  representational  transformation,  the  student  is  then  required  to 
transform  the  new  display  back  into  linguistic  form  again  when  he  oral- 
ly describes  his  work  to  his  teacher  and  peers.  Figure  7 shows  how  this 
transformation  process  works. 


OUTPUT 


Figure  7.  Representation  Transformation  (RETRAN)  Reading  Training  Procedure 


Development  of  Strand  II  Job  Reading  Passages 

In  order  for  the  students  to  practice  the  conceptualizing  and 
reading  skills  of  Strand  II,  there  was  a need  for  a body  of  materials 
which  the  students  could  use  in  making  representational  transforma- 
tions. Following  the  FLIT  approach,  this  need  led  to  the  development 
of  narrative  passages  that  were  designed  to  (1)  challenge  and  develop 
the  reading  skills  of  candidate  students,  but  not  be  as  difficult  as 
actual  technical  publications;  (2)  present  important  Job  concepts;  and 
(3)  lend  themselves  to  the  instructional  approach  of  making  transforma- 
tions. 


Selection  of  Strand  II  Content 


The  requirement  for  the  passage  to  present  job  concepts  led  to 
the  problem  of  identifying  which  concepts  should  be  presented  and  used 
in  the  JORP.  The  need  for  technical  information  to  be  presented  at  a 
more  general  and  less  difficult  level  than  that  of  actual  technical  pub- 
lications led  to  the  consideration  of  Career  Development  Courses  (CDCs). 

Basically,  CDCs  are  home  study  correspondence  type  material  that 
present  Job  knowledges  on  which  the  airman  must  demonstrate  competence 
before  progressing  within  a career  ladder.  An  analysis  showed  that  CDCs 
were  a good  resource  base  from  which  to  develop  Strand  II  materials. 
Furthermore,  Mockovak  (1974)  had  identified  the  major  reading  problem 
of  Air  Force  personnel  enrolled  in  base  reading  improvement  programs  as 
the  inability  to  read  and  understand  CDCs.  Therefore,  a methodology  was 
developed  for  selecting  content  areas  from  these  materials.  This  proce- 
dure is  presented  below  and  is  graphically  summarized  in  Figure  8.  A 
detailed  description  and  survey  of  the  way  "successful"  versus  "unsuc- 
cessful" students  dealt  with  the  CDC  volumes  is  presented  in  Appendix  A. 

Step  1 Determine  tasks  and  knowledge  requirements  for  each  AFSC  in 
the  Job  cluster  from  the  specialty  training  standard  (STS). 

Step  2 Determine  reference  materials  for  the  task  and  knowledge 
requirements  stated  in  the  STSs.  (AFM  50-23  states  that 
for  each  AFSC  in  the  job  cluster,  the  CDCs  are  the  source 
for  the  job  knowledge  components  of  the  Dual  Channel  OJT 
Program,  and  are  the  sole  reference  for  the  specialty 
knowledge  test  (SKT)  questions.  Since  the  STS  is  also  the 
controlling  document  for  the  SKT,  and  the  CDC  is  the  ref- 
erence for  the  SKT,  the  CDC  provides  a convenient  and  eco- 
nomical source  of  materials  from  which  to  develop  the  JORP 
materials.) 
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Obtain  CDCs  and  STSs  for  all  AFSCs  in  a career  cluster. 

Determine  those  CDC  objectives  that  were  tested  in  the 
course  final  examination  and  textual  references  for  the 
objectives.  This  step  reduces  the  volume  of  directly 
relevant  material  presented  in  the  CDC  that  needs  to  be 
considered  for  inclusion  in  the  training  program. 

Determine  major  content  areas  from  textual  references 
obtained  in  Step  4 across  AFSCs  in  the  JORP  cluster. 
Compare  content  areas  within  STS  to  ensure  focussing 
on  major  job  knowledges. 


Step  6 Match  each  CDC  objective  within  the  cluster  to  the  con- 
tent areas  specified  in  Step  5. 


Step  7 Specify  the  references  (text,  chapter  review  exercises, 
volume  review  exercises,  course  examination  questions, 
and  figures)  for  each  CDC  objective  identified  in  Step 
6 for  each  AFSC  in  the  JORP  cluster.  Steps  5,  6,  and  7 
further  reduce  the  CDC  source  materials.  The  product  was 
the  Cluster  Reference  Matrix.  The  matrixes  for  each  JORP 
cluster  are  shown  in  Appendix  B. 

Step  8 Develop  objectives  for  each  content  area  across  the  AFSCs 
within  the  JORP  cluster. 


Step  9 Write  JORP  reading  passages  using  cluster  reference  matrix 
and  objectives  of  Step  8 for  each  content  area. 


This  procedure  yielded  a cluster  of  general  content  areas  across  all  tha 
AFSCs  in  the  job  clusters.  The  content  areas  selected  in  this  manner 
formed  the  basic  resource  material  for  developing  and  writing  each  passage. 
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Figure  8.  Example  of  the  Method  for  Selection  of  Content  Areas  for  JORP  Clusters 


Writing  Guidelines 


Once  the  content  areas  and  their  references  had  been  identified, 
knowledge  objectives  for  each  were  specified.  The  knowledge  objec- 
tives specified  the  job  knowledges  that  should  be  achieved  by  the 
students  after  reading  and  comprehending  the  narrative  description 
for  a given  area. 

There  were  14  narrative  passages  prepared  to  represent  the  14 
Job  content  areas  identified  for  each  cluster.  Each  passage  is  ap- 
proximately 1,000  words  in  length  and  the  reading  grade  level  was 
targeted  at  the  9.0  to  10.0  RGL.  These  passages  were  also  written 
so  that  they  would  be  appropriate  for  making  representational  trans- 
formations from  printed  displays.  The  passage  title,  a short  descrip- 
tion of  the  content,  its  estimated  RGL,  and  the  total  number  of  words 
for  each  passage  in  the  two  J0RP  clusters  are  shown  in  Appendix  C.  • 

It  is  Important  here  to  note  the  distinction  between  the  teach- 
ing of  job  knowledges  and  the  teaching  of  reading.  Even  though  each 
passage  has  knowledge  objectives  and  presents  job  concepts,  it  is  not 
the  purpose  of  J0RP  training  to  teach  job  knowledges  that  are  needed 
to  perform  effectively  on  the  job.  This  is  the  mission  and  responsi-. 
billty  of  Air  Training  Command  and  the  Air  Force  On-The-Job  Training 
System.  But,  reading  training  cannot  be  completely  divorced  from  a 
content  area,  since  some  content  must  be  presented  with  which  to  con- 
duct reading  and  literacy  training.  Thus,  the  JORP  Strand  II  materi- 
els  use  job-relevant  content  to  reinforce  the  development  of  reading 
skills. 


Strand  II  Conceptualizing  Activities 

A workbook  containing  explanatory  and  practice  material  was  de- 
veloped to  introduce  the  students  to  the  concept  of  transformations. 
The  students  read  about  the  conceptual  activity  they  would  be  doing, 
then  practiced  the  activity  by  completing  a series  of  exercises. 
Following  this,  the  teacher  guided  the  group  discussion  and  peer 
group  interaction.  The  goal  of  this  "Tools  for  Learning"  workbook 
was  to  make  the  student  thoroughly  familiar  with  the  idea  of  repre- 
sentational transformation  and  to  provide  practice  in  transforming 
printed  displays  into  three  representation  forms:  pictures,  classi- 
fication tables,  and  flow  charts. 

Once  the  students  completed  all  the  exercises  in  the  "Tools  for 
Learning"  workbook,  they  then  practiced  these  skills  on  the  Strand 
II  narrative  passage  for  their  JORP  cluster. 


■i 
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Strand  II  job  concepts  were  presented  to  the  students  in  the 
form  of  job-reading  passages  to  which  they  applied  various  concep- 
tualizing strategies  or  schemes  useful  in  the  process  of  "compre- 
hending" the  concepts  contained  in  the  passage.  A conceptualizing 
task  in  the  JORP  was  defined  as  any  activity  in  which  the  student 
was  asked  to  transform  job  knowledge  information  from  one  repre- 
sentational form  (input)  to  another  (output).  The  training  materi- 
als focus  on  transforming  printed  linguistic  displays  into  other 
forms  of  representation  — iconic  and  schematic. 


Iconic  Representation  — In  this  type  of  conceptualizing 
activity,  the  student  was  required  to  represent  concepts  by  draw- 
ing pictures  of  them.  However,  it  was  stressed  that  this  activity 
did  not  require  "artistic  ability";  the  important  point  was  to  il- 
lustrate with  any  level  of  artistry,  the  concepts  presented  in  the 
job-reading  passage.  The  drawings  were  to  be  labeled.  Figure  9 
shows  different  student  iconic  transformations  from  the  same  nar- 
rative passage  describing  Aircraft  Main  Landing  Gear. 

Schematic  Representation  — Classification  Tables  — In  this 
type  of  conceptualizing  activity,  the  student  was  required  to  con- 
struct a classification  table  from  the  information  presented  in  the 
narrative.  The  students  were  to  construct  their  own  superordinate 
categories  and  sort  the  concepts  in  the  narrative  passage  into  the 
correct  cells  of  their  table.  Figure  10  shows  different  student 
classification  tables  for  the  same  narrative  passage  description. 

Schematic  Representation  — Flow  Charts  — In  this  type  of 
conceptualizing  activity,  the  student  was  required  to  transform  the 
printed  display  into  a schematic  display  called  a flow  chart.  Stu- 
dents were  taught  the  use  of  standard  symbols  in  the  "Tools  for 
Learning"  workbook,  and  used  these  to  show  sequential  activities  and 
decision  points  identified  in  the  procedural  narrative  descriptions. 
Figure  11  shows  different  student  flow  charts  for  the  same  narrative 
description  about  updating  Aircraft  Maintenance  Directives. 
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Figure  9.  Student  Examples  of  Iconic  Presentations  from  the  Same  Narrative  Passage  (cont) 


Figure  9.  Student  Examples  of  Iconic  Presentations  from 
the  Same  Narrative  Passage  (continued) 
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Igure  10.  StudenC  Examples  of  Classification  Tables  from  the  Same  Narrative  Passage 
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Figure  10,  Student  Euaples  of  Classification  Tables  f roe  the  Same  Narrative  Passage  (continued) 


Strand  II  Instructional  Approach 


The  design  and  nature  of  ttie  Strand  II  activities  demanded  active 
participation  and  interaction  by  the  teacher  with  the  students.  The 
training/learning  activities  were  designed  to  be  directed  and  con- 
trolled by  the  teacher.  There  was  less  reliance  on  the  materials  to 
"carry  the  instruction"  than  in  the  Strand  I modules.  Of  primary 
importance  in  the  Strand  II  activities  was  the  fact  that  students 
were  not  just  given  worksheets  to  "fill  out  and  turn  in."  Rather, 
after  learning  the  basics  of  making  transformations,  the  students 
were  given  a passage  and  then  guided  by  the  teacher  to  "transform" 
their  own  internal  conceptualization  of  the  written  passage  into  a 
different  representational  mode.  The  teacher's  role  was  to  provide 
assistance  as  the  students  made  the  desired  transformations. 

It  is  important  to  note  that  the  transformations  made  by  the 
students  were  neither  right  nor  wrong  in  the  sense  that  there  was 
only  one  "correct"  answer  as  in  Strand  I.  The  transformations  made 
by  the  students  were  judged  by  the  individual  student,  his  peers,  and 
the  instructor  in  terms  of  its  appropriateness  to  the  transformation 
task  instructions  and  the  content  presented  in  the  passage.  This 
judgement  was  arrived  at  through  group  and  individual  interactions 
with  the  teacher  about  the  representations  produced  by  the  students. 
The  teacher's  role  was  that  of  a catalyst,  to  stimulate  the  students 
to  examine  their  transformation  products.  It  was  through  this  type 
of  interaction  and  feedback  that  the  students  learned  to  increase 
their  conceptualizing  and  information-processing  skills. 


Chapter  3 
JORP  FIELD  TEST 


The  prototype  program  described  in  Chapter  2 was  field-tested 
at  Travis  Air  Force  Base,  California,  during  the  period  16  August 
1976  to  12  November  1976.  Travis  AFB  was  chosen  as  the  field  test 
site  because  it  met  the  following  criteria: 

e An  already  functioning  reading  program  with  adequate 
funding  and  appropriate  student-teacher  ratio 
(approx.  12:1). 

e A large  enough  population  in  the  relevant  AFSCs  to 
Insure  sufficient  sample  size. 

• Appropriate  classroom  facilities  for  an  individualized, 
self-paced  program. 

• Sufficient  major  command  interest  (Military  Airlift 
Command)  to  Insure  support  through  the  duration  of  the 
field  test. 

• Reasonable  proximity  to  HumRRO/Western  Division  offices 
on  the  Presidio  of  Monterey,  CA. 

The  existing  reading  program  at  Travis  Air  Force  Base  was  being 
conducted  primarily  with  VA  funds  through  the  veterans  Pre-Release 
Education  Program  (PREP),  which  was  administered  by  the  local  high 
school  district,  the  Travis  Unified  School  District.  The  classroom 
designated  for  JORP  consisted  of  a portion  of  a larger  room,  contain- 
ing 15  individual  carrels. 


SELECTION  OF  STUDENTS 

Actual  selection  of  the  airmen  who  were  to  participate  in  the 
JORP  was  done  through  the  base  education  office  using  criteria  iden- 
tical to  those  already  in  use  at  Travis  AFB,  and  directed  by  AFM  50- 
23,  to  wit:  trainees  who  have  a score  of  60  or  below  on  the  "general" 
aptitude  section  of  the  ASVAB  or  AQE  are  scheduled  for  USAFI  Achieve- 
ment Test  (UAT)  III,  Reading  section,  parts  1 and  2;  airmen  with  a 
reading  grade  level  less  than  9 on  either  part  1 or  part  2 are  en- 
rolled in  the  base  reading  improvement  program.  These  criteria  were 
to  be  used  to  identify  96  airmen  as  subjects  for  the  field  test;  how- 
ever, as  a result  of  the  accidental  use  of  inappropriate  criteria,  only 
61  persons  who  met  the  original  requirements  were  available  to  partici- 
pate in  the  JORP.  However,  32  airmen,  who  were  above  the  9th  RGL  and 
erroneously  selected,  were  kept  in  the  classes  for  a total  sample  of  93. 
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JORP  ASSESSMENT  INSTRUMENTS 


TVo  types  of  assessment  instruments  were  developed  for  use  in 
evaluating  the  JORP  prototype. 

1.  Strand  I in-training  module  mastery  tests  (proficiency 
tests) . 

2.  Overall  Strand  I & II  performance  tests  (Job-Oriented 
Reading  Program  Test). 


Strand  I Module  Proficiency  Tests 

In  order  to  permit  quality  control  monitoring  of  student  per- 
formance during  the  Strand  I training,  proficiency  tests  were  devel- 
oped for  each  of  the  modules  of  instruction.  These  were  used  to  as- 
sure that  the  student  had  developed  mastery  of  that  reading  task 

before  proceeding  to  the  next  module.  The  test  had  no  time  limit 
per  se;  however,  to  satisfactorily  master  the  task,  the  student  must 
have  met  the  dual  criteria  of  accuracy  and  time  — not  more  than  two 
wrong  answers  in  20  minutes  or  less. 

The  prototype  proficiency  test  (or  pro-test,  as  they  were  gener- 
ally referred  to)  is  made  up  of  two  sections,  each  with  its  own  text 

materials  and  set  of  ten  questions.  The  text  which  the  student  read 

in  order  to  answer  the  questions  was  a sample  of  job  reading  materials 
taken  directly  from  an  AF  publication,  and  the  questions  were  similar 
in  type  and  content  to  those  encountered  in  the  worksheets.  A separ- 
ate answer  sheet  was  provided.  The  student's  task  was  to  read  the 
questions  on  the  right  side  of  the  test  booklet,  look  up  the  answers 
in  the  pages,  of  text  provided  on  the  left  side  of  the  test  booklet, 
and  write  the  answer  on  the  separate  answer  sheet.  After  answering 
the  ten  questions  in  one  section,  the  student  turned  to  the  next 
section  of  the  test  booklet  and  continued  working.  Since  the  student 
might  have  to  take  more  than  one  test  within  a module  before  demon- 
strating mastery,  three  different  forms  of  the  test  were  developed 
for  each  module.  The  attempt  was  made  to  make  these  forms  equivalent. 

The  one  exception  to  thfc  above  was  in  the  Forms  module.  The 
Forms  module  did  not  have  an  overall  pro- test,  per  se.  Instead,  each 
form  had  its  own  set  of  three  alternate  tests  which  were  composed  of 
one  section  each.  The  instructions  for  completing  the  form  were  on 
the  left  side  of  the  test  booklet  and  the  information  to  be  filled 
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A. 


In  on  the  blank  form  was  on  the  right  side  of  the  booklet.  There 
were  approximately  20  entries  required  to  be  filled  in  on  the  form, 
but  the  accuracy  and  time  criteria  remained  the  same.  Otherwise, 
the  proficiency  tests  were  standard  across  the  various  modules  ex- 
cept for  the  job-related  content. 

Job-Oriented  Reading  Program  Test 

The  function  of  the  JORP  Test  was  to  provide  an  overall  measure 
of  the  ability  to  perform  the  basic  types  of  reading  tasks  encoun- 
tered in  learning  and  doing  various  Air  Force  jobs.  In  contrast  to 
the  module-specific  assessment  function  of  the  Strand  I module  pro- 
tests, the  JORP  Test  served  to  evaluate  the  effectiveness  of  the 
entire  program  in  teaching  what  it  was  designed  to  teach. 

The  test  required  students  to  read  actual  job  reading  materials 
to  obtain  the  kinds  of  Information  job  incumbents  seek  from  job  and 
training  manuals.  In  addition,  the  JORP  Test  measures  information 
processing  skills  by  requiring  the  student  to  transform  linguistic 
input  (print)  into  schematic  output  (flow  chart  and  classification 
table).  It  is  a group  test,  which  requires  one  hour  for  administra- 
tion and  consists  of  four  parts.  Parts  I and  II  measure  skills  at 
performing  typical  job  reading  tasks  in  the  Air  Force;  i.e.,  Strand 
I skills.  Parts  III  and  IV  measure  skills  at  making  representational 
transformations;  i.e.,  Strand  II  skills.  All  responses  are  of  the 
short  answer  or  fill-in  type,  which  substantially  eliminates  the  prob- 
lem of  chance  success.  There  were  three  alternate  forms  of  the  test: 
A,  B,  and  D.  The  JORP  Test  was  developed  to  meet  the  following  de- 
sign criteria: 

1.  Test  content  samples  directly  from  the  domain  of 
reading  skills  taught  in  the  JORP. 

2.  No  requirement  for  specific  job  knowledge. 

3.  Use  of  free,  constructed  responses,  rather  than  a 
fixed-alternative,  multiple-choice  answer  format. 

4.  A correct  answer  dependent  totally  upon  using  the 
Information  contained  in  the  test  passages  and  not  on 
guessing  or  prior  knowledge  of  the  content. 

The  test  also  served  to  provide  a summative  evaluation  instrument 
on  the  instructional  effectiveness  of  the  training  material  and  pro- 
cedures of  the  JORP. 
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CONDUCT  OF  THE  INSTRUCTIONAL  PROGRAM 


i 


Schedule 

The  instructional  day  at  the  Travis  JORP  was  divided  into  four 
2%  hour  sessions,  beginning  at  0800.  This  arrangement  allowed  four 
classes  per  day  to  be  held  in  the  same  facility  at  the  Travis  PREP 
center.  Students  were  dismissed  about  ten  minutes  before  the  end  of 
each  period,  to  allow  the  teacher  to  prepare  for  the  next  class. 

The  class  periods  of  two  hours  and  20  minutes  were  divided  be- 
tween Strand  I and  Strand  II  activities.  Although  the  relative  a- 
mount  of  time  devoted  to  each  varied  (on  some  days  the  whole  session 
would  be  spent  on  one  strand) , more  time  was  usually  spent  on  Strand 
I than  Strand  II.  Each  of  the  two  JORP  teachers  taught  two  classes 
per  day,  and'  each  class  ran  for  six  weeks. 


In-Processing 

On  the  first  day  of  class,  the  students  were  first  given  a 
short  orientation  to  the  purpose  of  the  reading  program  in  the  Air 
Force  and  the  activities  they  would  pursue  in  the  JORP.  Then,  art 
overall  program  pretest  (JORP  Test)  was  administered,  using  forms  A 
and  B alternately.  Following  the  JORP  Test,  students  filled  out  two 
background  information  questionnaires  — one  for  the  JORP  and  one 
for  the  Travis  Unified  School  District.  As  students  turned  in  their 
completed  questionnaires,  they  were  administered  their  first  pretest 
in  Strand  I. 

On  the  second  day  of  class,  the  students  were  administered  a 
general  reading  test.  After  the  first  two  classes  were  administered 
the  UAT  III,  it  was  discovered  that  many  of  the  students  had  taken 
the  UAT  III  several  times  before  enrollment  in  the  JORP  training.  As 
a result,  the  reading  comprehension  section  of  the  Test  of  Adult  Ba- 
sic Education  (TABE)  (CTB/McGraw  Hill,  1976)  was  used  to  get  an  esti- 
mate of  the  general  reading  comprehension  on  the  remaining  six  classes. 
The  TABE  Level  D was  administered  using  forms  3 and  A alternately. 
After  the  administration  of  this  test,  the  student  resumed  Strand  I 
activities  begun  the  day  before. 
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Strand  I 


After  the  JORP  Test  and  questionnaires  were  administered  the 
first  day,  the  student  was  assigned  to  the  Maintenance  or  Non- 
Maintenance  career  cluster  on  the  basis  of  his  or  her  AFSC.  The 
student  then  entered  one  of  the  four  modules  in  the  cluster  (chosen 
at  random  by  the  teacher).  As  Figure  12  shows,  the  student  progressed 
through  the  four  modules  of  Strand  I as  a result  of  satisfying  the 
Proficiency  Test  (pro-test)  passing  criteria. 

Upon  entering  their  first  module,  the  students  were  adminis- 
tered a pro-test  as  a pretest.  If  they  satisfied  the  dual  criteria 
(completion  within  20  minutes  and  a score  of  no  more  than  two  wrong) , 
they  advanced  to  another  module.  If  they  failed  the  pretest,  they 
were  given  a set  of  worksheets  to  complete.  In  the  Forms  module, 
each  worksheet  set  first  guided  the  student  step-by-step  through  the 
instructions  for  preparing  a particular  form,  then  provided  practice 
in  making  the  correct  entries.  In  the  other  modules,  there  were  two 
sets  of  worksheets.  Each  set  provided  practice  in  the  job-reading 
skill  of  that  module. 

In  all  cases,  the  criterion  for  successful  completion  of  a set 
of  worksheets  was  to  answer  every  question  correctly.  When  a stu- 
dent had  filled  in  one  worksheet,  he  handed  it  to  a peer  scorer,  and 
then  proceeded  to  the  next  worksheet  in  the  set.  The  scorer  indi- 
cated which  answers  were  incorrect  and  handed  the  worksheet  back  to 
the  student,  who  then  corrected  the  errors.  At  this  point,  the  stu- 
dent could  use  the  assistance  of  the  teacher  or  a peer.  The  work- 
sheet was  returned  to  the  student  as  many  times  as  necessary  to 

achieve  the  100%  criterion. 

When  the  student  had  achieved  100%  criterion  on  every  worksheet 
in  a set,  an  alternate  form  of  the  Proficiency  Test  was  administered. 
If  they  passed  this  test,  they  advanced  to  another  module.  If  they 
failed  it,  they  repeated  the  training  cycle,  using  the  same  set  of 
worksheets  (in  the  Forms  module),  or  a new,  second  set  (in  the  other 

three  modules)  and  then  were  given  a third  form  of  the  Proficiency 

Test. 


Ideally,  in  the  case  of  those  who  failed  the  pro-test  for  the 
third  time  (i.e.,  after  two  sets  of  worksheets),  tutoring  by  the 
teacher  or  a peer  would  be  followed  by  recycling  through  the  module 
until  the  pro-test  was  passed.  However,  in  actual  practice  a short- 
age of  time  obliged  the  teacher  to  advance  the  student  to  another 
module.  This  latter  course  was  necessary  to  insure  that  every  stu- 
dent received  training  in  every  module  of  Strand  I within  the  six- 
week  period  of  the  field  test. 
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Figure  12.  Flow  Chart  of  Student  Progression  Through  the  Four  JORP  Modules 


Strand  II 


For  the  first  two  to  three  weeks  of  each  six-week  class,  45  to 
90  minutes  per  day  were  used  to  complete  the  Strand  II  workbook, 
"Tools  for  Learning",  Group  sessions  were  conducted  by  the  teacher, 
during  which  the  three  conceptualizing  activities  were  practiced  and 
discussed.  Students  often  worked  in  small  groups  to  make  the  repre- 
sentational transformations  required  by  the  worksheets  in  the  work- 
book. To  discuss  a student's  work,  the  class  was  shown  the  worksheet 
or  it  was  reproduced  on  the  chalkboard. 

During  the  final  three  to  four  weeks  of  the  class,  the  job- 
reading passages  were  used  each  day  to  practice  a particular  mode  of 
transformation.  In  this  way,  about  14  passages  could  be  discussed 
during  the  course.  A typical  group  session  lasted  one  hour,  begin- 
nig  with  the  oral  reading  of  the  passage  for  the  day,  and  ending 
with  the  discussion  of  the  representational  transformations  produced 
during  the  session.  Ordinarily,  Strand  II  was  practiced  after  Strand 
I,  but  this  order  was  sometimes  reversed  for  variety. 

Out-Processing 

On  the  next  to  last  day  of  class,  each  student  took  the  JORP 
Test  form  (A  or  B)  not  previously  taken.  After  the  JORP  Test,  the 
general  reading  test  was  administered,  again  using  the  alternate 
form  for  each  student. 

On  the  last  day  of  class,  a Student  Post  Job-Oriented  Reading 
Program  Survey  was  given  to  each  student  to  complete.  After  this 
questionnaire  was  returned,  the  students  were  administered  an  AFHRL- 
developed  alternate  form  JORP  test  (Form  D)  parallel  to  those  devel- 
oped by  the  contractor  (HumRRO) . Finally,  each  student  was  given  a 
certificate  of  course  completion  during  a short  interview  with  the 
teacher.  At  this  interview,  the  student's  progress  in  the  JORP,  as 
measured  by  his  pre-to-post  JORP  Test  gain,  was  discussed. 
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Chapter  4 

EVALUATION  OF  THE  JORP  PROTOTYPE 

In  this  chapter  we  will  present  data  on  the  characteristics  of 
the  JORP  students  and  on  the  effectiveness  of  the  prototype  reading 
training  program  as  it  was  developed  and  operated  at  Travis  AFB,  CA. 
Summative  data  will  be  presented  primarily  in  terms  of  gains  and  end- 
of-course  achievement  levels  on  both  the  measure  of  general  reading 
comprehension  and  the  JORP  Test  measure  of  proficiency  in  performing 
a variety  of  job-reading  tasks. 

CHARACTERISTICS  OF  THE  JORP  STUDENT 

The  appropriateness  and  operating  effectiveness  of  any  training 
program  is  in  large  part  a function  of  the  students  who  are  engaged 
in  the  program.  This  section  describes  some  characteristics  of  the 
93  students  who  were  trained  at  Travis  AFB,  CA  during  the  field  eval- 
uation. 

All  of  the  students  were  assigned  to  Travis  AFB  on  a permanent 
party  basis  and  were  performing  in  various  job  assignments.  Detailed 
descriptive  data  were  obtained  from  a background  questionnaire  ad- 
ministered on  the  first  day  of  class. 

The  ages  and  educational  levels  of  the  students  are  shown  in 
Table  3.  The  median  age  was  22,  and  median  years  of  education  com- 
pleted was  12.  Ninety-five  percent  of  the  students  reported  having 
a high  school  diploma,  with  the  rest  having  a GED  equivalency,  which 
contrasts  with  their  mean  entry  RGL  of  9.24  on  measures  of  general 
reading  comprehension,  as  shown  in  Table  4. 

The  reading  comprehension  section  of  the  UAT  III  and  TABE  were 
used  to  measure  the  general  reading  comprehension  of  the  JORP  students. 
Two  classes  (N  * 23)  were  pre-  and  post-tested  using  UAT  III,  while 
(for  reasons  stated  in  Chapter  3,  In-Processing,  page  57 ) the  rest 
of  the  classes  (n  * 70)  were  pre-  and  post-tested  using  the  TABE. 

Since  the  difference  between  the  entry  RGL  means  on  the  UAT  III 
and  TABE  was  not  significant,  they  were  combined.  Thus,  in  the  re- 
mainder of  this  report,  whenever  measurement  of  general  reading  com- 
prehension is  discussed  and/or  compared,  it  will  be  based  on  the  com- 
bined student  RGLs. 
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TABLE  4.  ENTRY  READING  GRADE  LEVEL  (RGL)  OF  JORP  STUDENTS 


RGL 

UAT 

N 

111 

X 

TAKE 

N X 

COMBINED 

N . X 

CUMULATIVE 

X 

0 

0.0 

1 

1.4 

1 

1.1 

1.1 

0 

0.0 

3 

4.3 

3 

3.2 

4.3 

2 

8.7 

5 

7.1 

7 

7.5 

11.8 

7.0-  7.9 

5 

21.7 

6 

8.6 

11 

11.8 

23.6 

8.0-  8.9 

7 

30.5 

16 

22.9 

23 

24.7 

48.3 

9.0-  9.9 

3 

13.1 

11 

15.7 

14 

15.1 

63.4 

10.0-10.9 

1 

4.3 

11 

15.7 

12 

12.9 

76.3 

11.0-11.9 

5 

21.7 

9 

12.9 

14 

15.1 

91.4 

12.0-12.9 

5 

7.1 

5 

5.4 

96.8 

> 12.9 

3 

4,3 

3 

3.2 

100.0 

TOTAL 

23 

100.0 

70 

100.0 

93 

100.0 

Mean  RGL 

8.83 

9.38 

9.24 

63 


. 

L 


\s  noted  in  Chapter  3,  61  students  were  identified  as  reading 
below  the  9.0  RGL  on  the  basis  of  UAT  III  scores.  However,  the  re- 
sults of  the  second  day  in-processing  testing  on  the  general  reading 
test,  showed  that  only  45  of  the  93  students  scored  below  the  9.0  RGL. 
This  result  reflects  the  difficulties  encountered  in  identifying  ap- 
propriate students  for  a developmental  reading  course  of  this  type. 


There  were  only  eight  female  students  in  the  sample  of  93.  The 
distribution  of  enlisted  grades  of  the  students  is  shown  in  Table  5. 


TABLE  5.  ENLISTED  GRADE  OF  JORP  STUDENTS 


GRADE 


E-l 

1 

1.1 

E-2 

26 

27.9 

E-3 

26 

27.9 

E-4 

29 

31.2 

E-5 

11 

11.9 

TOTAL 

93 

100.0 

There  was  some  variety  in  the  ethnic  group  representation  in 
the  JORP  sample,  as  shown  in  Table  6.  Only  12%  of  the  sample  were 
foreign  born.  Of  these,  73%  were  from  the  Philippine  Islands  and 
their  mean  time  in  the  United  States  was  32.25  months. 


TABLE  6.  ETHNIC  GROUPS  OF  JORP  STUDENTS 


ETHNIC  GROUP 

N 

% 

Black  American 

43 

46.: 

Anglo  American 

34 

36. 

Filipino 

8 

8.i 

Spanish-American 

3 

3.: 

Pacific  Island 

2 

2.: 

American  Indian 

2 

2.: 

Puerto  Rican 

1 

i.: 

TOTAL 

93 

100.1 

The  language  background  of  the  JORP  students  is  shown  in  Table  7. 

For  the  most  part,  English  was  the  primary  language  of  the  students, 
and  only  9%  had  potential  English  As  A Second  Language  (ESL)  difficulties. 
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TABLE  7.  LANGUAGE  BACKGROUND  OF  JORP  STUDENTS 


LANGUAGE 


Students ' 

Primary 

Language 


Language 
Spoken 
in  Home 


NX  NX 


English 

Filipino 

Spanish 

Samoan 

Navaho 

German 

Guamanian 


85  91 

5 6 

1 1 

1 1 

1 1 

0 0 

0 0 


79  85 

7 8 

3 3 

1 1 

1 1 

1 1 

1 1 


TOTAL 


93  100  93  100 


Whatever  the  literacy  difficulties  of  the  JORP  students,  they  were 
not  likely  due  to  ESL  problems,  but,  more  likely,  to  inadequate 
reading  skills  in  their  own  primary  language. 

The  distribution  of  Air  Force  specialties  (AFSs)  in  terms 
of  the  training  cluster  for  the  JORP  sample  is  shown  in  Table  8.* 
There  were  23  different  AFSs  represented  in  the  sample.  The  Air- 
craft Maintenance  Specialist  (431)  career  specialty  made  up  roughly 
one-third  of  the  sample. 

*It  is  interesting  to  note  that  the  10  career  fields  that  Mockovak 
(1974)  cited  as  accounting  for  83. 9X  of  the  individuals  enrolled 
in  a reading  program  at  the  time  of  his  survey  were  the  same  career 
fields  that  accounted  for  81. 8X  of  the  students  enrolled  in  the  JORP 
Field  Tryout  at  Travis  AFB. 
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Administration 

702  £ 

Printinq 

71 1 1 

Personnel 

m 2 

Security  Police 
81 1 2 

Medical 


2 6 14.2 

1 1 2.4 


Eighty  percent  of  the  students  were  technical  school  graduates. 
Those  students  who  were  enrolled  in  a CDC  concurrently  with  enroll- 
ment in  the  JORP  training  made  up  273!  of  the  sample.  Self-reported 
performance  with  CDC  materials  for  all  students  is  shown  in  Table  9. 
Most  (83Z)  of  these  students  reported  that  they  were  able  to  complete 
their  CDC  requirements  without  any  difficulties. 


TABLE  9.  CDC  PERFORMANCE  OF  JORP  STUDENTS 


Failed 

a vol.  review 
exercise 


Failed 

end-of-course 

examination 


N 

X 

N 

X 

Yes 

10 

11 

16 

17 

No 

83 

89 

77 

83 

TOTAL 

93 

100 

93 

100 

The  students  were  asked  whether  or  not  they  did  any  Job  reading 
and  if  that  type  of  reading  presented  any  problems  to  them.  Table 
10  summarizes  the  responses.  It  is  interesting  to  note  that  the  22 
percent  of  Air  Force  personnel  reporting  difficulties  with  job  print- 
ed materials  is  consistent  with  the  percent  of  Navy  job  incumbents 
(25Z)  reporting  "some"  difficulty  understanding  job  printed  materials 
in  another  study  (Sticht,  Fox,  Hauke,  & Zapf,  1976). 
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matter  In  the  Air  Force.  The  median  dally  job  reading  time  for  the 
JORP  students  was  l.S  hours,  which  is  comparable  to  the  median  dally 
job  reading  time  of  1.8  hours  reported  by  Navy  job  Incumbents  (Stlcht 
et  al.,  1976).  When  queried  about  their  desire  to  take  the  course, 
S9Z  said  they  wanted  to  take  the  JORP  training,  while  36%  said  they 
did  not;  5%  were  undecided. 


STRAND  I MODULE  EVALUATION  DATA 

In  this  section,  data  appropriate  to  the  evaluation  of  the  In- 
dividual modules  of  Strand  I are  presented.  The  purpose  of  Strand 
I Is  to  provide  training  In  the  performance  of  specific  reading  tasks 
Involved  in  learning  about  how  to  do  a job  and  in  the  day-by-day  per- 
formance of  that  job.  This  objective  was  given  full  operational  mean- 
ing In  the  sets  of  proficiency  tests  constructed  for  each  module.  The 
evaluation  of  the  Strand  I program  consisted,  then,  of  determining  the 
effectiveness  of  each  module  In  training  students  to  reach  the  mastery 
criteria  of  the  crlterla-ref erenced  proficiency  tests  for  that  module. 


Training  Effectiveness  of  Modules 

The  four  Instructional  modules  of  Strand  I were  Schematics,  Nar- 
rative, Procedural  Directions,  and  Forms.  For  the  first  three  of 
these,  there  were  module  proficiency  tests  prepared  in  three  alternate 
forms,  which  were  used  to  assess  module-specific  proficiency  before 
and  after  Instruction.  The  Forms  module  does  not  have  an  overall 
module  pre-  and  post-test  for  proficiency;  rather,  each  of  the  four 
forms  used  for  training  Is  Its  own  "sub-module"  with  three  alternate 
versions  for  each  form.  Table  12  shows  for  each  module  and  form: 

1.  The  percentage  of  students  who  successfully  passed 
the  pretest  for  the  module  or  fora  and  advanced 
laedlately  to  the  next  module. 

2.  The  percentage  who  felled  the  pretest  and  then  passed 
the  post  test  after  Instruction. 

3.  The  percentage  of  students  who  failed  the  pretest  and 
subsequently  felled  the  post  test  or  never  did  complete 
the  module. 

Data  from  both  the  maintenance  and  non-maintenance  clusters  have  been 
combined  In  this  table. 
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TABLE  12.  STRAND  I MODULE  PERFORMANCE 


ALL  JORP  STUDENTS 


(1) 

Module 

(2) 

N 

(3) 

Passed 

Pretest 

% 

(A) 

Passed 
Post- 
test % 

(5) 

Failed  or 
Did  Not 
Complete 
% 

(6) 

Training 

Effectiveness* 

% 

Schematic 

92 

12 

28 

60 

32 

Narrative 

91 

10 

21 

69 

23 

Procedural 

Directions 

92 

17 

32 

51 

39 

Forms 

AFTO  781A 

85 

12 

7A 

1A 

84 

AFTO  781H 

78 

0 

88 

12 

88 

AF  601b 

82 

5 

82 

13 

86 

AF  1996 

67 

0 

93 

7 

93 

It  is  clear  from  Table  12  that  most  of  the  students  needed  train- 
ing in  all  of  the  modules.  For  example,  while  17%  of  the  students  were 
able  to  reach  criterion  level  on  the  Procedural  Directions  pretest 
(and  thus,  did  not  need  training  in  this  task  area),  none  of  the  stu- 
dents were  able  to  reach  the  criterion  level  in  filling  out  AF  Form 
1996  prior  to  instruction. 


Regarding  the  Failed  or  Did  Not  Complete  column  (5),  it  should  be 
recalled  that  some  students  did  not  achieve  proficiency  in  a given  mod- 
ule after  recycling,  and  they  were  moved  into  the  next  module  to  make 
sure  they  had  some  exposure  to  all  the  different  job-reading  tasks  be- 
fore the  six  weeks  of  training  were  completed.  While  this  practice 
precluded  strict  adherence  to  mastery  performance  criteria,  the  modules 
are  not  necessarily  hierarchical  and  so  cumulative  deficits  in  skills 
would  not  be  expected  to  result. 


* (A)  7 (A  + 5) 
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The  effectiveness  of  each  training  module  is  shown  in  column  6. 
These  data  were  obtained  by  dividing  the  entry  in  column  4 by  the 
sum  of  the  entries  in  columns  4 and  5.  The  training  effectiveness 
percentage  is  thus  the  percentage  of  those  students  who  could  not 
pass  the  module  proficiency  test  before  instruction,  but  who  did 
pass  the  pro-test  in  that  module  or  form  after  instruction. 

The  overall  effectiveness  of  the  Forms  module  was  approximately 
three  times  that  of  the  other  three  instructional  modules.  The  rea- 
son for  this  dramatic  difference  lies  in  the  nature  of  the  task  be- 
ing trained.  In  the  Schematic,  Narrative,  and  Procedural  Directions 
modules,  the  students  practiced  the  generalized  task  of  extracting 
information  embedded  in  various  informational  displays  (i.e.,  table, 
graphs,  narrative  descriptions,  flow  charts,  etc.).  The  students 
were  pre-  and  post-tested  on  the  reading  tasks  in  these  modules 
using  informational  displays  similar  to  those  practiced  during  in- 
struction. However,  with  each  form,  the  students  practiced  filling 
out  only  that  particular  form,  rather  than  practicing  the  generalized 
task  of  filling  out  forms;  they  were  pre-  and  post- tested  on  the 
same  form  used  during  instruction,  although  different  information 
with  which  to  fill  in  the  blanks  was  provided.  Simply  put,  the  job- 
reading tasks  required  of  the  students  to  meet  criterion  levels  on 
form-specific  training  in  the  JORP  were  different  from  those  required 
in  the  other  modules. 

The  training  effectiveness  of  the  Schematic,  Narrative,  and 
Procedural  Directions  modules  in  training  students  to  the  specified 
level  of  proficiency  was  32,  23,  and  39  percent,  respectively.  These 
relatively  low  values  suggest  that,  for  about  two- thirds  of  the  JORP 
students,  the  approximately  42  hours  of  instruction  in  the  Strand  I 
type  tasks  may  have  been  insufficient  time  in  which  to  accomplish  the 
required  learning.  In  this  regard,  Sticht  (1975)  reports  approximately 
twice  the  training  effectiveness  for  the  FLIT  training  program  which 
devoted  approximately  twice  the  training  time  to  Strand  I type  tasks 
than  did  JORP.  Of  course,  a variable  that  may  have  had  some  impact 
here  is  low  motivation  of  those  JORP  students  who  expressed  resent- 
ment at  being  required  to  participate  in  the  field  test. 

In  spite  of  these  considerations,  it  does  appear  that  the  JORP 
students  did  acquire  job-related  reading  skills. 
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Speed  & Accuracy  Criteria 


Satisfactory  performance  on  the  module  proficiency  tests  is 
specified  in  terms  of  the  dual  criteria  of  accuracy  and  time:  90% 
of  the  20  constructed  response/fill-in  questions  must  be  answered 
correctly  within  20  minutes  for  students  to  pass  a module  proficiency 
test.  Table  13  shows  the  percentage  of  students  failing  module  pre- 
tests for  reason  of  insufficient  accuracy,  taking  too  much  time,  or 
for  reasons  of  both  time  and  accuracy.  For  example,  of  the  students 
who  failed  the  Schematics  pretest  and  entered  instruction  in  that 
module,  56%  failed  to  meet  the  accuracy  criterion,  7%  failed  to  meet 
the  time  criterion,  and  the  remaining  37%  failed  to  meet  both  criteria. 
On  the  average,  failure  to  attain  the  90%  level  of  correct  answers 
accounts  for  38%  of  the  student  failures  on  the  module  pretests  and 
46%  of  the  students  failed  to  meet  either  the  accuracy  or  the  time 
criterion. 

For  those  same  students.  Table  14  shows  their  mean  pretest  scores 
on  the  criterion  measure  they  failed.  This  table  shows  that  students 
failing  the  Schematics  module  pretest  on  accuracy  only,  answered  69.2% 
of  the  questions  correctly  on  the  pretest.  Those  failing  to  meet  the 
time  criterion  took  an  average  of  25.7  minutes  to  complete  the  pretest, 
and  those  with  sub-standard  scores  on  both  accuracy  and  time,  answered 
61.6  of  the  questions  correctly  and  took  29.7  minutes  to  finish  the 
test.  Data  are  shown  only  for  the  variables  on  which  performance  was 
sub-standard:  students  whose  less-than-90%  correct  scores  are  shown  in 
the  accuracy  column,  did  complete  the  test  in  not  more  than  20  minutes; 
those  whose  failing  time  scores  are  shown  in  the  time  column  did  get  at 
least  90%  of  the  test  items  correct;  and  those  whose  average  scores  on 
both  criterion  dimensions  are  shown,  failed  on  both  accuracy  and  time. 

In  summary,  the  data  in  Table  12  indicate  that  most  students 
failed  to  pass  the  module  pretests;  in  Table  13,  we  see  that  38%  of 
the  students  failing  the  module  pretests  failed  to  meet  the  90%  cri- 
terion level  of  accuracy,  and  another  46%  failed  on  both  accuracy  and 
time;  and  those  students  who  failed  the  module  pretests  averaged  67% 
correct  on  those  pretests.  Thus,  it  seems  reasonable  to  conclude  that, 
while  many  JORP  students  enter  with  some  ability  to  perform  these  job- 
reading tasks,  they  do  show  a need  for  additional  training  on  the 
fundamental  skills  trained  in  the  JORP. 
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TABLE  13.  PERCENTAGE  OF  STUDENTS  FAILING  A MODULE  PRETEST  FOR 
REASONS  OF  ACCURACY,  TIME,  & BOTH  ACCURACY  & TIME. 


X OF  STUDENTS  FAILING  PRETEST  CRITERIA 


1 

• 

Module 

Accuracy  X 

Time  X 

Accuracy  & Time  X 

Schematics 

56 

7 

37 

Narrative 

18 

28 

54 

1 

Procedural 

Directions 

42 

12 

46 

Mean 

38 

16 

46 

Mean  N - 80 


TABLE  14.  MEAN  PRETEST  SCORES  OF  STUDENTS  FAILING  MODULE  PRETESTS 
Module  Reason  for  Pretest  Failure 


Accuracy 

X correct 

Schematics 

69.2 

Narrative 

68.0 

Procedural 

Directions 

62.6 

Mean 

66.71 

N - 80 

Time 

minutes 

Accuracy  & 
X correct 

Time 

minutes 

25.7 

61.6 

29.7 

31.4 

69.3 

31.3 

27.2 

63.0 

27.9 

29.74 

65.16 

29.77 

74 
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SUMMATIVE  EVALUATION  OF  THE  JORP 


This  section  of  the  report  deals  with  the  overall  effect  of  the 
two  separate  Instructional  strands.  Overall  effectiveness  Is  based 
on  the  pre-  and  post-test  administration  of  alternate  forms  of  the 
JORP  Test  discussed  earlier  and  the  general  reading  comprehension 
tests  (UAT  III  and  TABE).  The  results  of  these  tests  are  shown  In 
Table  15.  The  pre-to-post  gain  on  the  JORP  Test  is  significant  be- 
yond the  .001  level.  There  was  approximately  a 50%  Increase  In  the 
test  score  after  Instruction.  However,  for  the  general  reading  meas- 
ure, the  value  was  not  significant,  and  the  .4  gain  In  RGL  could  be 
accounted  for  by  chance  fluctuation  in  the  sampling  or  by  testing  ar- 
tifacts. Because  the  JORP  Test  had  not  been  scaled  with  a general 
reading  test  at  the  time  of  this  field  test,  RGLs  are  not  reported 
for  the  number  of  points  obtained  by  the  students.  Because  of  this 
consideration,  AFHRL  has  undertaken  a norming  study  of  the  JORP  Tests. 
For  purposes  of  the  present  study,  it  was  possible  to  get  an  estimate 
of  what  a certain  number  of  points  on  the  JORP  Test  means  In  terms  of 
RGL  by  computing  a least  squares  regression  of  RGL  on  JORP  Test  scores 
using  the  preliminary  norming  data. 

The  resulting  correlation  between  general  reading  grade  level 
and  JORP  Test  scores  was  .52. 


TABLE  15.  READING  PERFORMANCE  OF  JORP  STUDENTS 


TYPE  OF  MEASURE 

N 

ENTRY 

EXIT 

GAIN 

VALUE 

P 

JORP  Test 

93 

33.40* 

49.53* 

16.13* 

7.90 

<.001 

General  Reading 

UAT  III  & TABE 

93 

9.24** 

9.65** 

.41** 

1.47 

NS 

♦Number  of  points  obtained  out  of  80  possible. 
♦♦Reading  Grade  Level 
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But  using  this  correlation,  a pretest  value  of  33.40  on  the  JORP  Test 
converts  to  a 10.15  RGL  and  the  post  test  value  of  49.53  on  the  JORP 
Test  converts  to  a 11.32  RGL.  Thus,  the  estimated  gain  from  pre  to 
post  on  the  JORP  Test  in  terms  of  RGLs  was  1.17,  or  nearly  three  times 
the  gain  made  in  general  reading  scores.  This  is  the  same  gain  ratio 
that  Sticht  (1975)  reports  for  the  FLIT  students.  For  the  sake  of 
clarity,  and  since  the  general  reading  measures  did  not  show  signifi- 
cant gains  for  the  JORP  students,  the  sub-group  comparisons  reported 
in  the  remainder  of  this  chapter  are  based  only  on  the  pre-to-post 
gains  on  the  JORP  Test. 


JORP  Test  Part  Score  Gains 

The  JORP  Test  is  composed  of  four  parts,  with  parts  1 and  2 
measuring  the  overall  effects  of  Strand  1,  and  parts  3 and  4 measuring 
the  overall  effects  of  Strand  II.  Table  16  shows  the  pre-to-post  scores 
for  each  part  separately  and  for  Strand  I and  Strand  II  components. 

There  were  significant  pre-to-post  gains  on  every  part  of  the  JORP  Test. 
However,  there  were  significantly  more  gains  made  on  the  Strand  II  part 
than  the  Strand  I part;  in  fact,  twice  as  much  gain.  This  difference 
could  be  accounted  for  in  terms  of  motivation  to  learn  Strand  II  skills, 
since  students  reported  on  their  post  JORP  training  surveys  that  Strand 
II  activities  were  preferred  and  more  useful  to  them  than  Strand  I ac- 
tivities. (See  Appendix  D,  Student  Post  JORP  Survey,  Questions  #10  and 
#12,  for  greater  detail.  The  results  of  the  students'  post  training 
survey  of  attitudes  will  be  discussed  as  they  relate  to  comparisons  made 
on  the  JORP  Test  between  various  groups.  Appendix  D shows  the  complete 
student  survey  data.)  This  pre-to-post  gain  on  the  JORP  Test  holds  for 
nearly  all  the  following  comparisons  between  sub-groups  and  supports  the 
position  that  the  instruction  received  by  the  students  was  effective  and 
relevant  to  those  job-reading  skills  measured  by  the  JORP  Test. 


TABLE  16.  OVERALL  JORP  TEST  RAW  SCORE  CORRECT; 
PRE  AND  POST  INSTRUCTION 


JORP  TEST  PART 

STRAND 

STRAND 

1 

2 

3 

4 

I 

II  T 

P 

Mean 

Pretest 

8.14 

9.09 

9.66 

6.57 

17.23 

16.23  .802 

NS 

Mean 

Posttest 

10.99 

11.57 

15.57 

11.01 

22.56 

26.58  3.124 

<.01 

Gain 

2.85 

2.48 

5.91 

4.44 

5.33 

10.36 

T 

3.709 

3.051 

7.950 

5.089 

4.111 

8.362 

P 

<001 

<01 

» \ 

o 

o 

M 

<001 

<.001 

<001 

N = 93 


Sub-Group  Comparisons  of  JORP  Test  Effectiveness 

In  this  section,  we  present  data  on  various  sub-group  comparisons 
and  the  effectiveness  of  the  instruction  on  these  groups.  Because  of 
the  initial  difficulties  in  identifying  students  below  the  9.0  RGL  for 
entry  in  the  training  program,  there  were  a large  number  of  students 
reading  at  or  above  that  level.  Table  17  shows  the  comparison  of  these 
two  groups  in  terms  of  JORP  Test  scores.  The  students  were  divided  in- 
to the  above-  and  below-9.0  RGL  groups  on  the  basis  of  their  performance 
on  the  general  reading  comprehension  measure  administered  the  first  day 
of  class.  Table  17  shows  that  students  above  the  9.0  RGL  had  a signif- 
icantly higher  pretest  score  on  the  JORP  Test.  This  may  be  due  to  their 
higher  general  reading  skills  and  abilities.  While  there  were  also  sig- 
nificant posttest  differences  between  the  RGL  groups,  these  were  prob- 
ably due  to  the  differences  in  the  initial  reading  skills  and  abilities 
between  the  two  groups.  Both  groups  did  show  significant  improvement  in 
JORP  Test  performance. 


TABLE  17.  PRE  AND  POST  JORP  TEST  SCORES  FOR 
STUDENTS  ABOVE  & BELOW  9.0  RGL 


<9.0 

— 9.0 

T 

P 

N 

Mean 

SD 

N 

Mean 

SD 

Pretest 

45 

27.58 

12.08 

48 

38.87 

12.14 

4.50 

.001 

Posttest 

45 

45.44 

13.75 

48 

53.33 

14.25 

2.71 

.01 

T 

6.55 

T 

5.35 

P < .001 

P 

< .001 

Since  the  current  project  was  focused  on  the  development  of  two 
prototype  training  clusters,  the  Air  Force  was  concerned  with  how 
effective  these  two  training  curricula  were  in  meeting  the  needs  for 
job  reading  training  across  different  AFSs.  Because  a variety  of 
students  with  different  AFSs  formed  the  JORP  sample,  it  was  possible 
to  look  at  the  differences  in  performance  between  the  groups  for  whom 
the  training  curricula  content  had  been  designed,  and  all  other  stu- 
dents in  the  program  for  whom  the  content  was  not  specifically  designed. 
Therefore,  all  students  coming  from  the  64,  70,  42,  and  43  career  fields 
were  compared  to  all  other  students  in  the  sample.  Table  18  presents 
these  data. 

TABLE  18.  PRE  & POST  JORP  TEST  SCORES  OF  STUDENTS  FOR  WHOM 
TRAINING  MATERIALS  WERE  DESIGNED  VERSUS  ALL 
OTHER  STUDENTS 


64,  70,  42,  43*  ALL  OTHERS 

(Designed  For)  (Not  Designed  For) 


N 

Mean 

SD 

N 

Mean 

SD 

T 

P 

Pretest 

55 

34.18 

13.62 

38 

32.28 

12.96 

.68 

NS 

Posttest 

55 

52.07 

12.97 

38 

45.08 

15.57 

2.28 

.05 

T 

7.05 

T 

3.89 

P 

< .001 

P 

< .001 

♦See  Table  8 for  designation  of  AFS  codes. 
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From  Table  18,  it  can  be  seen  that  both  groups  made  significant 
gains  on  their  pre-to-post  scores,  indicating  that  the  instruction 
was  effective  for  both  groups.  However,  the  Designed  For  group  (stu- 
dents coming  from  the  64,  70,  42,  and  43  career  fields)  achieved  sig- 
nificantly greater  post  scores  than  did  the  Not  Designed  For  group. 

This  suggests  that,  for  the  training  curricula  to  be  maximally  effec- 
tive in  training  job-reading  skills,  the  curriculum  content  should  be 
appropriate  to  the  job  which  the  airman  is  performing.  The  exact  rea- 
sons for  this  finding  are  unclear.  One  possible  explanation  involves 
the  further  practice  on  the  job  of  reading  skills  and  the.  integration 
of  knowledges  taught  in  the  reading  training  school.  Another  possibil- 
ity is  that  the  students  for  whom  the  materials  were  designed  were  more 
motivated  because  the  materials  were  more  appropriate  to  them.  However, 
the  important  point  remains,  that  the  job-reading  training  curriculum 
needs  to  be  designed  for  the  AFSs  that  comprise  the  training  cluster  in 
order  to  achieve  maximum  effectiveness  in  a job-reading  training  pro- 
gram. 


This  finding  was  also  supported  by  many  critical  comments  (both 
anecdotal  and  as  reported  in  the  follow-up  surveys)  concerning  the  in- 
appropriateness  of  the  curriculum  for  those  JORP  students  for  whom  the 
training  curriculum  was  not  designed.  However,  it  is  interesting  to 
note,  again,  that  the  job-reading  skill  performance  of  this  group  did 
significantly  improve  as  a result  of  training  when  pre-to-post  scores 
are  compared  within  this  group. 

Table  19  presents  the  data  from  a comparison  of  all  Maintenance 
students  and  all  Non-Mair.tenance  students  on  the  JORP  Test.  (See  Table 
8 for  AFS  designations  of  Maintenance  and  Non-Maintenance  groups.) 


TABLE  19.  PRE-TO-POST  JORP  TEST  SCORE  FOR  ALL  MAINTE- 
NANCE & ALL  NON-MAINTENANCE  STUDENTS 

MAINTENANCE  NON-MAINTENANCE 


N 

Mean 

SD 

N 

Mean 

SD 

T 

P 

Pretest 

51 

32.90 

13.08 

42 

e 

34. 38 

13.63 

.531 

NS 

Post  Test 

51 

50.14 

14.05 

42 

48.79 

15.16 

.442 

NS 

T 6.41 
P < .001 


T 4.58 
P < .001 


r 
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Again,  the  pre-to-post  gains  within  both  groups  were  significant, 
but  there  was  no  significant  difference  between  the  groups  or.  pre  or 
post  scores.  Thus,  the  training  appears  to  have  been  equally  effective 
for  both  the  Maintenance  and  Non-Maintenance  groups. 

Comparisons  were  made  within  each  training  cluster  between  the 
students  whose  AFS  was  specific  to  the  content  sources  from  which  the 
training  materials  were  drawn -and  all  other  students  who  took  that 
particular  training  cluster.  These  data  are  shown  in  Tables  20  and  21. 


TABLE  20.  COMPARISON  OF  PRE-TO-POST  JORP  TEST  SCORES  OF 
MAINTENANCE  STUDENTS  FROM  THE  42  & 43  CAREER 
FIELDS  TO  ALL  OTHER  MAINTENANCE  PERSONNEL* 


MAINTENANCE  (42,43)  MAINTENANCE  (All  Others) 

N Mean  SD  N Mean  SD  TP 


Pretest 

43 

33.97 

13.26 

8 

25.88 

10.60 

1.90 

NS 

Post  Test 

43 

52.09 

13.08 

8 

37.63 

13.57 

2.78 

.01 

T 

6.38 

T 

1.93 

P 

.001 

P 

NS 

*See  Table  8 for  designation  of  AFS  codes. 




t 

TABLE  21;  COMPARISON  OF  PRE-TO-POST  JORP  TEST  SCORES  OF  NON- 
MAINTENANCE STUDENTS  FROM  CAREER  FIELDS  64  & 70  TO 
ALL  OTHER  NON -MAINTENANCE  PERSONNEL* 


NON-MAINT.  (64,  70)  NON-MAINT.  (All  Others) 


N 

Mean 

SD 

N 

Mean 

SD 

T 

P 

Pretest 

12 

34.92 

15.43 

30 

34.00 

13.15 

.182 

NS 

Post  Test 

12 

50.67 

13.34 

30 

48.03 

15.97 

.547 

NS 

T 

2.675 

T 

3.71 

P 

.02 

P 

.001 

*See  Table  8 for  designation  of  AFS  codes. 


80 


1 


As  shown  in  Table  20,  the  only  group  for  whom  the  training 
did  not  produce  a significant  increase  from  pce-to-post  training 
was  in  the  group  of  Maintenance  personnel  who  were  not  from  the 

42  or  43  career  fields.  This  lack  of  significance  was  probably 
due  to  the  size  of  the  group  (N  = 8)  since  the  direction  of  change 
was  in  the  anticipated  direction.  The  differential  effect  of  the 
training  shown  in  Table  20  for  the  Maintenance  group  from  the  42 
and  43  career  field  is  supportive  of  the  differential  effects  of 
the  training  shown  in  Table  18,  since  the  curriculum  materials  for 
the  Maintenance  cluster  came  exclusively  from  the  content  of  42  and 

43  career  fields. 

A comparison  of  Technical  School  graduates  and  Non-Technical 
School  student  scores  is  presented  in  Table  22. 


TABLE  22.  PRE-TO-POST  JORP  TEST  SCORES  FOR  TECHNICAL 
SCHOOL  GRADUATES  & DIRECTED  DUTY  ASSIGNEES 


TECH.  SCHOOL  DIRECTED  DUTY 

GRADUATES  ASSIGNEES 


N Mean  SD  N Mean  SD 


Pretest 

74 

34.07 

13.83 

19 

30.84 

11.06 

0.941 

NS 

Post  Test 

74 

50.73 

14.75 

19 

44.84 

12.72 

1.593 

NS 

Again  the  pre-to-post  gains  after  instruction  were  significant 
within  both  groups.  However,  there  were  no  significant  differences 
between  the  groups  in  terns  of  JORP  scores. 

In  sumnary,  the  data  Indicate  that  the  JORP  training  resulted 
in  increased  proficiency  at  job-reading  skills  for  all  groups  of 
students.  The  relatively  low  initial  scores  of  both  high  and  low 
ability  readers  suggests  that  general  reading  level  may  not  be  a 
good  indicator  of  job-reading  ability.  However,  the  more  proficient 
readers  tend  to  achieve  better  scores  after  training  on  the  job- 
reading tasks  than  do  the  less  apt  students.  This  finding  is  con- 
sistent with  evidence  from  educational  research  in  which  the  more 
able,  higher  ability  students  normally  get  more  out  of  their  train- 
ing experience  than  do  the  low  ability  students. 

There  was  evidence  presented  that  suggests  the  content  of  the 
training  curriculum  is  an  important  and  potent  variable.  Influencing 
the  effectiveness  of  the  training  system.  Careful  consideration 
should  be  given  to  the  manner  in  which  future  Job-reading  training 
curriculum  clusters  are  generated  and  developed  for  the  Air  Force. 

Detailed  attltudlnal  data  on  the  effectiveness  of  the  JORP 
training  for  both  students  and  their  supervisors  are  presented  in 
Appendixes  D and  E respectively. 
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Concluding  Conments 


During  the  course  of  this  project,  several  issues  surfaced 
which  seemed  appropriate  to  address  as  part  of  this  report. 

An  important  consideration  in  the  implementation  of  any  pro- 
gram of  the  type  involved  here  is  the  appropriateness  of  various 
delivery  systems,  and  the  likelihood  of  each  being  successful.  Dur- 
ing the  field  test,  several  problems  that  arose  were  specifically 
related  to  the  lockstep  nature  of  the  instructional  program.  Ini- 
tial scheduling  problems  were  encountered,  and  there  was  consider- 
able inconvenience  to  those  whose  shifts  changed  during  the  course 
of  training,  causing  them  to  work  more  than  the  normal  8 hour  day, 
sometimes  in  non-contiguous  time  periods.  In  an  operational  setting, 
a self-paced  learning  center  approach  may  be  a more  desirable  delivery 
system.  While  this  was  not  possible  within  the  constraints  of  the 
field  test,  the  current  set  of  materials  should  be  compatible  with 
such  an  approach.  Strand  I is  essentially  self-paced,  and  the  in- 
structor could  handle  all  the  test  scoring  that  during  the  field 
test  was  shared  with  the  students.  While  Strand  II  was  specifically 
intended  to  capitalize  on  the  benefits  of  peer  interaction,  the  ef- 
fectiveness of  the  materials  would  probably  not  diminish  substantially 
if  used  within  a learning  center  environment.  Students  could  study 
the  "Tools  for  Learning"  workbook  and  then  apply  the  techniques  to 
the  specially  constructed  passages.  If  there  happened  to  be  more 
than  one  student  in  the  center  working  on  Strand  II,  they  could  com- 
pare and  criticize  each  other's  work  in  much  the  same  manner  as  was 
originally  intended.  If  such  interaction  was  not  possible,  a set  of 
flow  charts,  pictures,  and  classification  tables  which  had  previously 
been  produced  by  others  could  be  used  for  comparison  by  the  individual 
student.  While  this  approach  would  require  somewhat  more  tedious  at- 
tendance to  the  task  and  is  not  the  most  desirable,  one  could  expect 
that  considerable  benefit  would  be  derived.  No  new  materials  would 
have  to  be  developed  for  a learning  center  approach;  one  would  merely 
have  to  collect  several  sets  of  Strand  II  student  products  to  be  used 
for  a student's  self-evaluation. 

Even  though  those  students  whose  AFSC's  were  not  specific  to  the 
available  JORP  clusters  generally  demonstrated  significant  improve- 
ments in  post-test  JORP  scores,  it  still  seems  reasonable  to  suggest 
that  job-related  content  is  an  important  aspect  of  the  curriculum 
materials.  This  is  true  both  conceptually  and  practically.  One  com- 
ponent of  reading  ability  is  familiarity  with  job  concepts  and  vocab- 
ulary. To  the  extent  to  which  the  students  can  acquire  improved 
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knowledges  in  these  two  areas,  they  should  become  more  capable  on 
the  job.  Also,  to  the  extent  that  most  of  the  content  of  the  mater- 
ials is  familiar  or  consistent  with  their  current  knowledge,  their 
attention  can  be  directed  specifically  to  the  reading  tasks  being 
trained.  Anecdotal  information  as  well  as  responses  to  questions 
on  the  post-JORP  student  surveys  suggest  that  one  of  the  primary 
criticisms  of  the  course  by  some  students  was  that  it  was  not  re- 
lated directly  to  their  specific  job.  Thus,  it  seems  reasonable  to 
state  that  the  following  two  criteria  are,  at  the  least,  desirable 
in  the  future  implementation  of  the  JORP  concept:  (1)  reading  tasks 
taught  in  the  program  should  be  relevant  to  job  task  demands,  and 
(2)  the  content  of  the  materials  within  which  these  skills  are  trained 
and  practiced  should  facilitate  maximum  transfer  to  the  job  and,  thus, 
be  specific  to  the  AFSCs  of  those  enrolled  in  the  reading  program. 

With  respect  to  this  latter  point,  it  is  unrealistic  and  unneces- 
sary to  go  to  the  extreme  of  requiring  immediate  implementation  of  one 
package  of  JORP  materials  for  each  career  field.  This  is  certainly 
not  necessary  in  cases  of  closely  related  fields,  such  as  the  42  and 
43  career  fields  in  this  study.  However,  a close  look  at  the  reading 
task  demands  of  the  many  AF  career  areas  would  be  required  to  make 
precise  estimates  of  the  number  of  clusters  that  are  needed.  In  con- 
sidering this  issue,  a suggested  approach  would  be  to  develop  materials 
for  only  those  segments  of  the  Air  Force  population  in  which  reading 
problems  appear  to  be  most  severe.  As  noted  earlier  (page  14) , 
ten  career  fields  appear  to  contribute  somewhat  more  than  eighty  per- 
cent of  the  student  enrollment  in  base  level  reading  improvement  pro- 
grams. It  is  our  suggestion  that  these  career  fields  be  addressed 
first  if  the  full  operational  implementation  of  JORP  is  to  be  pursued. 
Specifically,  it  is  recommended  that:  (1)  the  present  nonmaintenance 
cluster  be  split  into  a 70/73  administrative  cluster  and  a 64  supply 
cluster,  and  (2)  three  additional  clusters  be  developed  to  cover  the 
54/55  civil  engineering,  81  security  police,  and  60  transportation 
areas.  This  would  provide  a total  of  six  clusters  for  full  JORP  im- 
plementation. These  JORP  clusters  - maintenance,  administration, 
supply,  civil  engineering,  security  police,  and  transporation  - would 
address  most  of  the  problem  readers  in  the  Air  Force.  The  currently 
available  maintenance  and  nonmaintenance  JORP  clusters  could  be  used 
until  these  more  specific  additional  clusters  are  developed. 

The  final  and  perhaps  most  significant  consideration  in  review- 
ing the  JORP  is  that  of  validity.  The  selection  of  tasks  to  be  included 
in  the  program  was  not  based  on  the  results  of  a systematic  reading  task 
analysis.  Rather,  it  was  partially  based  on  the  general  knowledge  by 
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the  investigators  of  reading  tasks  encountered  on  the  job  and  partially 
on  the  identification  of  reading  tasks  from  the  CDC  materials.  Both 
of  these  approaches  are  open  to  some  criticism,  the  former  because  of 
its  subjectivity  and  the  latter  because  thorough  reading  of  the  CDCs 
Is  not  necessary  to  pass  the  CDC  End  of  Course  Examination.  Thus  there 
is  mainly  speculation  that  the  skills  taught,  particularly  In  Strand  II 
would  be  useful  in  the  performance  of  actual  reading  tasks  required  by 
the  Job.  While  these  shortcomings  of  the  JORP  do  exist,  they  are  cur- 
rently being  addressed  and  it  is  anticipated  that  future  reports  will 
document  the  findings  of  this  follow-on  research.  In  spite  of  these 
limitations  it  still  seems  appropriate  to  recognize  the  face  validity 
of  the  curriculum  materials  and  to  recommend  that  the  Air  Force  con- 
tinue to  move  toward  a functional  literacy,  job-oriented  reading  ap- 
proach, as  opposed  to  the  general  literacy  training  that  has  tradition- 
ally been  used. 
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APPENDIX  A 


CAREER  DEVELOPMENT  COURSES  (CDC) 


DESCRIPTION  OF  CDC 

In  the  Air  Force,  the  upgrade  training  (UGT)  has  been  highly  structured 
to  ensure  that  airmen  have  both  the  job  skills  proficiencies  and  job  knowl- 
edges to  perform  satisfactorily  on  the  job.  This  dual  track  — one  of 
skill  proficiencies  and  the  other  of  job  knowledges  — is  incorporated 
into  the  on-the-job  training  (OJT)  system. 

CDCs  are  self-study  correspondence  materials  administered  by  the 
Extension  Course  Institute  (ECI)  at  Gunter  AFB,  Alabama.  For  nearly  every 
AFSC  in  the  Air  Force,  there  is  a set  of  CDC  materials  that  has  to  be  com- 
pleted as  the  airman  advances  in  skill  within  an  AFS.  The  Dual  Channel  OJT 
program  requires  that  airmen  training  to  the  next  higher  skill  level  through 
formal  UGT  use  the  CDC  self-study  courses  geared  to  their  AFS. 

Typically,  a CDC  is  made  up  of  three  or  four  volumes.  Each  volume  con- 
tains a series  of  chapters  composed  of  specific  objectives,  followed  by  nar- 
rative text  and  then  a series  of  questions.  Chapter  Review  Exercises  (CRE) , 
to  guide  the  student's  study.  At  the  completion  of  each  volume,  the  student 
completes  a Volume  Review  Exercise  (VRE)  which  is  an  open-book  multiple- 
choice  exercise  on  the  information  in  the  volume. 

When  the  VRE  answer  sheet  is  completed,  it  is  forwarded  to  ECI  for 
grading  and  the  student  proceeds  to  the  next  volume.  VRE  scores  are  re- 
turned to  the  student  indicating  which  objectives  in  the  volume  need  fur- 
ther study.  Upon  satisfactory  completion  of  all  the  VREs,  the  student  is 
then  scheduled  to  take  the  Course  Examination  (CE)  at  the  Base  Testing 
Office.  This  examination  is  closed-book,  and  serves  to  determine  whether 
or  not  a trainee  has  satisfactorily  completed  the  CDC.  After  completing 
both  the  job  skill  proficiency  checkouts  and  the  CDC  course  exams,  the 
airman  is  then  considered  for  upgrade  action  to  the  next  highest  skill 
level  within  his  AFS.  (See  AFM  50-23  for  a more  complete  discussion  of 
CDCs.) 
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SURVEY  OF  STUDY  SKILLS  WITH  CDCs 


Mockovak  (1974)  reported  In  his  survey  of  reading  programs  In  the 
Air  Force  that,  of  the  5,774  airmen  enrolled  at  the  time  of  the  survey, 

55Z  were  cited  as  having  difficulty  reading,  comprehending,  and  passing 
CDC  material.  Since  the  successful  completion  of  CDCs  Is  prerequisite  to 
advancement  In  the  Air  Force,  It  seemed  useful  to  attempt  to  determine  If 
there  were  certain  study  skills,  learning  strategies,  etc.,  that  distin- 
guished trainees  who  were  "successful"  and  "unsuccessful"  on  their  CDCs. 

It  was  thought  that  this  might  reveal  skills  that  might  be  trained  in  the 
JORP.  This  appendix  presents  the  findings  of  that  survey. 

CDC  Survey  Sites  and  Student  Characteristics 

Travis  and  McClelland  AFBs  were  chosen  as  sites  to  conduct  the  CDC 
survey  because  of  their  proximity  to  the  contractor  and  the  availability 
of  a sufficient  sample  of  airmen  in  the  431  and  702  career  fields.  A 
successful  student  was  defined  as  any  trainee  who  had  passed  any  CE  for 
any  CDC  on  the  first  attempt.  An  unsuccessful  student  was  defined  as  any 
trainee  who  had  failed  any  CE  for  any  CDC  on  the  first  attempt.  A total 
of  37  students  were  Identified  for  the  sample.  Table  A-l  shows  a summary 
of  the  student  characteristics. 

There  were  basically  no  differences  between  the  successful  and  un- 
successful students  on  any  of  the  characteristics  shown  in  Table  A-l,  with 
the  exception  that  the  unsuccessful  students  had  more  time  in  the  career 
field  and  more  rank. 
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TABLE  A-l.  DESCRIPTIVE  CHARACTERISTICS  OF  CDC  SURVEY  INTERVIEWEES 


Student  , Successful Unsuccessful  TOTAL 


Characteristics 

N 

Z 

N 

Z 

N 

X 

Source  of 

• 

Tech. School 

13 

35.1 

14 

37.9 

27 

73.0 

3-skill 

Training 

DDA 

5 

13.5 

5 

13.5 

10 

27.0 

career 

702  Admin 

24.3 

9 

24.3 

18 

48.6 

Field 

Assigned 

431  A/C  Main. 

24.3 

10 

27.1 

19 

51.4 

Grade 

E-2 

1 

2.7 

1 

2.7 

2 

5.4 

E-3 

16 

43.2 

15 

40.6 

31 

83.8 

E-4 

1 

2.7 

2 

5.4 

3 

8.1 

E-5 

— 

— 

1 

2.7 

1 

2.7 

Education 

11 

_ 

4 

10.8 

4 

10.6 

Level 

12 

14 

37.9 

12 

32.4 

26 

70.3 

(Highest 

13 

3 

8.1 

2 

5.4 

5 

13.5 

grade  com- 

14 

- 

- 

1 

2.7 

1 

2.7 

pleted. ) 

15 

1 

2.7 

** 

1 

2.7 

Time  in 

4 Mos-1  Yr 

15 

10 

27.1 

25 

67.6 

career 

1-2  yrs 

2 

5.4 

5 

13.5 

7 

18.9 

Field 

2-4  yrs 

1 

2.7 

3 

8.1 

4 

10.8 

Over  4 yrs 

"■* 

*■* 

1 

2.7 

1 

2.7 
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Findings  of  COC  Survey 

Each  student  was  interviewed  using  a structured  Interview  form. 

The  interview  took  approximately  45  minutes  to  complete.  Table  A-2  shows 
the  amount  of  time  the  students  spent  studying  their  CDCs.  Few  of  the 
students  ~ either  successful  or  unsuccessful  — had  a regular  study 
schedule.  When  they  did  study,  both  groups  studied  about  the  same  amount 
of  time. 

TABLE  A-2.  STUOY  TIME  DEVOTED  TO  COMPLETED  CDC  MATERIALS 


Successful  Unsuccessful 


Regular  Study  Schedule  Yes 

3 

3 

No 

15 

14 

Number  of  Days  Range 

1-5 

1-5 

Studied  During  Week  Avg 

3.2 

3.0 

Number  of  Hours  Range 

1/2-6 

1/2-5 

spent  in  each  study  Avg 

session 

2.2 

2.5 

Table  A-3  shows  the  kinds  of  techniques  reportedly  being  used.  Also, 
Table  a-3  shows  comparison  data  for  a group  of  Navy  students  with  similar 
characteristics  and  time  in  service.  Both  Air  Force  groups  are  remarkably 
similar  in  the  techniques  they  employ.  Also,  the  frequencies  of  use  of 
the  techniques  by  the  Navy  and  Air  Force  personnel  are  similar.  The  cor- 
relation between  the  assigned  ranks  of  the  Air  Force  and  Navy  personnel 
based  on  the  frequency  of  use  of  the  study  techniques  shown  in  Table  A-3 
is  .61.  Underlining  was  the  most  frequent  study  technique  used,  with 
outlining  and  drawing  as  the  least  frequent  methods  used  by  both  the 
Air  Force  and  Navy  personnel.  Thus,  those  techniques  that  require  more 
internal  thought  processing  by  the  student  were  least  preferred. 
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TABLE  A-3,  STUDY  TECHNIQUES  EMPLOYED  BY  AIR  FORCE  & NAVY  STUDENTS 

Q.  When  you  study,  do  you  use  any  of  these  study  techniques? 


Air  Force 
Frequencies 

N - 37 
Total 

Air  Force 

N - 61* 
Total 

Navy 

Success  Unsuccess 

X Rank 

X Rank 

Make  Outline 

2 

1 

8 

7 

40 

8 

Underline  Important  Parts 

14 

15 

78 

1 

81 

1 

Draw  Pictures  & Diagrams 

1 

1 

5 

8 

52 

7 

Look  up  words  in  Dictionary 

7 

9 

43 

2.5 

67 

5 

Look  over  section  before  study 

9 

5 

16 

6 

68 

4 

Ask  yourself  questions 

8 

4 

37 

4 

78 

2 

Take  Notes 

4 

2 

32 

5 

72 

3 

Try  to  memorize 

8 

8 

43 

2.5 

62 

6 

*from  Sticht,  Fox,  Hauke,  and  Zapf , 1976  (r~m  .61) 


The  students  were  asked  to  describe  their  general  CDC  study  approach 
by  responding  to  a question  designed  to  identify  the  sequence  of  study 
activities  engaged  in  by  the  airmen.  Table  A-4  lists  the  alternative  re- 
sponses available  to  the  students,  rank  ordered  in  terms  of  decreasing 
study  time  likely  to  be  required  by  the  use  of  that  study  sequence. 


TABLE  A-4.  CDC  GENERAL  STUDY  SEQUENCE  FOLLOWED  BY 
SUCCESSFUL  & UNSUCCESSFUL  STUDENTS 


1.  Preview  volume,  then  read  word-for-word, 
complete  CREs  & VREs  (ECI  prescribed 
approach) . 

2.  Preview  volume,  read  word-for-word, 
complete  VREs. 

3.  Read  only  parts  of  volumes  necessary  to 
answer  CRE  & VREs. 

4.  Skim  read  through  volume,  then  read  only 
parts  necessary  to  answer  VREs. 

5.  Read  only  parts  of  volume  necessary  to 
answer  VREs. 

6.  Did  not  study  or  did  not  read. 


Successful  Unsuccessful 


91 


7 

0 


2 

4 


Both  the  successful  and  unsuccessful  group  frequencies  are  nearly  identi- 
cal for  the  first  four  study  sequences.  The  last  two  sequences  (5  and  6) 
show  some  interesting  results.  More  than  three  times  the  number  of  suc- 
cessful students  concentrated  only  on  the  parts  of  the  CDC  volume  neces- 
sary to  answer  the  VREs.  Also,  21%  of  the  unsuccessfuls  did  no  study  or 
reading  of  the  CDC. 

It  is  interesting  to  note  that  sequence  5 is  the  most  effecient  method 
to  simply  get  by  the  hurdle  of  completing  CDC  requirements.  It  appears 
that  the  successful  students  were  able  to  perceive  this  fact  and  adjust 
their  study  habits  to  achieve  this  goal.  This  interpretation  is  further 
supported  by  the  data  in  Table  A-5,  showing  the  students'  estimates  of  the 
number  of  CE  questions  that  come  from  the  VRE  questions. 

TABLE  A-5.  PERCEIVED  & ACTUAL  PERCENTAGE  OF  CE  QUESTIONS 
COMING  FROM  THE  VRE  QUESTIONS 

Q.  In  your  estimation,  what  percentage  of  the  questions  on  the  final 


course  exam  (CE)  come  from  the  Volume  Review  Exercise  (VRE)  questions? 


Percent  of  students  estimating  90% 
or  higher. 

Successful 

61% 

Unsuccessful 

32% 

Actual  percentage  of  CE  questions 
from  VRE  questions. 

Admin . 

100% 

A/C  Maint. 
70-90% 

Thus,  more  successful  than  unsuccessful 

students  do  not 

read  all  the 

information  in  the  CDC  volume,  but  do  only  the  minimum  amount  of  study  re- 
quired to  do  the  VRE  exercises  and  concentrate  on  them.  This  study 
sequence  is  rewarded  by  the  fact  that  in  both  career  fields  surveyed,  the 
percentages  of  CE  questions  that  come  almost  word-for-word  from  the  VREs, 
is  more  than  sufficient  to  satisfactorily  complete  the  CDC  material,  since 
the  passing  criterion  is  60%  for  these  two  career  development  courses. 
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The  information  displayed  in  Table  A-6  suggests  that  most  of  the  stu- 
dents do  not  perceive  the  CDC  material  as  providing  much  of  their  job 
knowledge.  The  students  indicated  their  main  source  of  job  knowledge 
came  from  doing  the  job. 

TABLE  A-6.  SOURCES  OF  JOB  KNOWLEDGE 

Q.  Where  did  you  get  MOST  of  your  job  knowledge?* 


Source 

Successful 

Unsuccessful 

Technical  Training  School 

2 

3 

Job  Experience 

12 

15 

Supervisor 

4 

7 

Co-Worker 

6 

6 

CDC  Materials 

— 

2 

Previous  Work  Experience 

1 

1 

*Some  students  reported  two  or  more  sources. 

Table  A-7  shows  the  students'  perceived  problems  with  the  CDC  materials. 
The  items  in  the  Problems  column  were  presented  in  the  survey  for  the  stu- 
dents to  check  as  they  desired.  The  most  significant  problem  cited  by  the 

TABLE  A-7.  PROBLEMS  WITH  CDC  MATERIALS 


PROBLEMS 

Successful 

Unsuccessful 

Total 

Material  not  related 
to  job 

5 

4 

9 

Reading  Problems 
(vocabulary , compre- 
hension, questions) 

7 

7 

Materials  were  boring 

5 

1 

6 

Not  enough  time  allowed 
to  complete  study 

- 

1 

1 

Could  not  remember 
information 

93 

1 

1 

I m 

L.  _ 


unsuccessful  students  was  that  of  reading  difficulties.  This  finding  is 
consistent  with  Mockovak's  (1974)  data.  The  successful  students'  comments 
centered  about  the  boring  nature  of  the  CD C materials  and  their  lack  of 
relevance  to  the  job  assignment. 
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APPENDIX  B 

AIR  FORCE  MAINTENANCE  & NON-MAINTENANCE  CLUSTER  TOPIC  CHARTS 


These  charts  show  the  references  to  the  major  content  areas  selected 
for  the  Inclusion  In  the  JORP.  The  following  descriptions  define  the 
column  headings  of  the  charts. 

CONTENT  AREAS 

This  column  represents  the  common  concepts  and  topic  headings  across- 
AFSCs  within  each  cluster  from  which  the  passages  were  developed  and 
reading  exercises  delineated. 


AFSC 


This  is  the  Air  Force  Specialty  Code  designation  for  those  CDC 
materials  Included  in  each  cluster.  The  exact  titles  are  as  follows: 


Maintenance: 


42152  - 
43121A  - 

43151C  - 

43151F  - 
43151E  - 


Aircraft  Fneudraulic  Repairman 

Aircraft  Maintenance  Specialist, 
Engine  Aircraft 

Aircraft  Maintenance  Specialist, 
One  & Two  Engine 

Aircraft  Maintenance  Specialist, 

Aircraft  Maintenance  Specialist, 
Two  Engines 


Reciprocating 

'Jet  Aircraft, 

Turboprop  Aircraft 
Jet  Aircraft  Over 


Non-Maintenance : 

64550  - Inventory  Management  Specialist 
64750  - .Materiel  Facilities  Specialist 
70250  - Administration  Specialist 

OBJECTIVE  (NO.) 


This  column  la  the  Behavioral  Objective  number  or  the  Lesson  or 

» 

Study  number  given  in  the  CDC  materials. 
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OBJECTIVE  (REF.) 

• This  column  shows  the  volume  and  page  reference  where  the  objective 
number  can  be  located. 

TEXT  DISCUSSION 

This  column  lists  the  volume,  paragraph,  and  page  references  for 
the  textual  presentation  and  discussions  within  each  CDC. 

CHAPTER  REVIEW  EXERCISES 

This  column  lists  the  references  for  and  number  of  items  (questions) 
in  the  chapter  review  exercises  associated  with  the  content  area  within 
the  CDC  materials. 

VOLUME  REVIEW  EXERCISES 

This  column  lists  the  references  for  and  number  of  items  (questions) 
that  make  up  the  volume  review  exercises.  The  questions  are  all  four- 
alternative,  multiple-choice  type  items. 

CE  ITEM  NUMBER 

This  column  shows  the  Course  Examination  (CE)  numbers  of  the  CE 
questions  which  are  matched  to  the  VREs  and  objectives  covering  the 
content  area  listed, 

CE  QUESTION  TYPE 

This  column  shows  the  type  of  the  Course  Examination  questions 
referenced  in  the  previous  column.  All  CE  questions  are  classified  on 
the  basis  of  the  following  three  categories: 

Perceptual  - The  stem  and  the  correct  alternative  closely  match 
the  word-by-word  form  presented  in  a single  sentence  of  the  text. 
This  type  of  question  may  appear  in  an  inverted  order  from  that 
found  in  the  text.  Also,  this  question  type  could  be  answered 
on  the  basis  of  the  orthographic  features  of  the  text. 
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Example : 

CE  Question 

The  Three  authorized  Inspection  concepts  used  In  the  Air  Force  are 

a.  periodic,  intermediate,  & phased 

b.  crew.  Intermediate,  and  major  & minor 

c.  periodic,  phased,  and  major  & minor 

d.  periodic,  phased,  and  dock 

CPC  Text 

1-3.  Inspection  Concepts.  Authorized  inspection  concepts  are 
(1)  periodic,  (2)  phased,  and  (3)  minor  and  major.  (These 
concepts  will  be ) 

Linguistic  - There  is  no  paraphrasing  of  substantive  words  from 
the  question  to  the  textual  presentation.  The  word  order  may  or 
may  not  be  Inverted  from  the  question  to  the  text  and  key  elements 
of  the  answer  to  the  question  are  separated  by  one  or  more  sentences 
in  the  text. 

Example; 

CE  question 

24.  During  aerial  cargo  delivery,  the  pilot  signals  to  drop  the 
load  by 

a . voice  communication 

b.  sounding  a horn 

c.  illuminating  a green  light 

d.  illuminating  a red  light 

CPC  Text 

16-31.  Aerial  Delivery  Systems.  The  term  "aerial  delivery" 
as  used  in  this  chapter  means  the  airdrop  of  cargo  and  per- 
sonnel. Cargo  type  . . . The  pilot  or  copilot  sounds  the  horn 
as  the  drop  zone  is  neared.  He  illuminates  the  red  light  as  a 
signal  to  get  ready  to  drop  the  load.  Green  light  illumination 
in  the  signal  to  drop  the  load. 

Semantic  - Must  involve  some  or  all  paraphrasing  of  substantive 
words  and/or  requires  prior  knowledge  regarding  the  subject  area  in 
order  to  answer  the  question. 
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Example 


CE  Question 

2.  A red  diagonal  line  penciled  on  an  aircraft  maintenance  form 
Indicates  that 

a.  the  aircraft  Is  unsafe  for  flight 

b.  compliance  with  an  urgent  action  TCTO  Is  overdue 

c.  a condition  exists  that  may  restrict  use  of  the  aircraft 

d.  a required  scheduled  Inspection  has  not  been  completed 

CPC  Text 

d.  The  red  diagonal  Indicates  a condition  that  does  not  cause 
the  aircraft  to  be  grounded.  To  clear  the  aircraft  for  flight, 
either  the  pilot  or  the  maintenance  officer  must  sign  a release. 

FIGURES,  TABLES,  CHARTS,  ETC. 

This  column  provides  the  references  for  all  the  figures,  charts, 
tables,  etc.  that  were  presented  and  referred  to  in  the  textual  nar- 
rative covering  the  content  area  in  the  CDC  materials. 

STS 


This  column  shows  the  Speciality  Training  Standard  references  that 

\ 

match  the  content  area  to  the  appropriate  section  of  the  STSs  for  the 
Maintenance  Cluster  AFSCs. 
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41132 
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APPENDIX  C 

STRAND  II  JOB  READING  MANUAL  TOPICS 
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This  appendix  presents  detailed  responses  to  an  attitudinal  sur- 
vey of  JORP  students1  post  training.  The  entire  question  and  its 
associated  scale,  and  the  responses  of  the  sample  are  shown.  In  the 
left  margii  is  shown  the  percent  oi  students  who  failed  to  respond  to 
that  question.  The  total  student  sample  was  93.  Where  the  question 
calls  for  an  open-ended  answer,  responses  have  been  categorized  and 
shown  at  the  end  of  this  appendix,  referenced  by  question  number. 
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We  are  trying  to  evaluate  the  effects  of  the  JORP  Training  Course  and 


need  your  help.  Since  you  are  one  of  the  first  graduates  of  the  course, 
we  ask  that  you  fill  out  the  attached  survey  honestly  and  that  your  coa- 
ments really  reflect  your  opinions  or  feelings.  The  Information  you 
provide  will  help  us  determine  how  well  the  JORP  course  is  meeting  the 
needs  of  the  Air  Force,  and  will  be  kept  confidential. 


In  accordance  with  paragraph  30,  AFR  12-35,  Air  Force  Privacy  Act 
Program,  the  following  information  about  this  survey  is  provided: 

a.  Authorities 

Title  10  USC,  Section  8012,  Secretary  of  the  Alr'Force:  Powers 
and  Duties  Delegation  by;  Executive  Order  9397,  22  Nov  1943, 
Numbering  System  for  Federal  Accounts  Relating  to  Individual 
Persons . 

b.  Principal  Purpose 

This  survey  is  being  conducted  to  obtain  evaluation  data  on  tha 
effectiveness  of  the  Job-Oriented  Reading  Program  (JORP)  developed 
by  the  Human  Resources  Research  Organisation  under  contract  to  the 
Air  Force  Human  Resources  Laboratory  at  Lowry  AFB,  Colorado. 

c.  Routine  Use 

The  survey  data  will  be  analyzed  to  provide  specific  recommenda- 
tions for  program  changes  in  the  course  prototype.  All  data 
will  be  kept  completely  confidential. 

d.  Disclosure 


Disclosure  is  voluntary.  No  adverse  action  of  any  kind  may 
be  taken  against  any  individual  who  elects  not  to  participate 
in  the  survey. 


USAF  SCN  7T24 

Sep,  1976  thru  Dec,  1976 


Percent  PLEASE  ANSWER  THE  FOLLOWING  QUESTIONS. 
Not  Responding 

^ 1.  Did  you  like  the  JORP  training  course? 


5.4 

37.6 

36.6 

14.0 

5.4 

Liked  it  Liked  it. 
very  much. 

WHY? 


Neither  liked 
nor  disliked. 


Disliked 

it. 


Disliked  It 
very  much. 


1.0  2.  Do  you  think  the  tasks  you  practiced  in  the  JORP  course  apply  to 

the  reading  tasks  you  do  on  the  job? 


9.7 


35.5 


23.6 


27.9 


2.2 


Very  Most  things  Most  things  Does  not  Can't  tell 

definitely  apply.  do  not  apply,  apply  at  if  the  course 

apply.  all.  applies  or  not. 


3.7 

7.5 

2.2 

4.3 


5.4 


3.  Do  you  feel  the  JORP  course  has  helped  you  on  the  job?  38.7  yes  58.1  MO 

WHY?  

In  other  reading  tasks?  67.8  yES  ^ ^ NO 

4.  Did  you  have  enough  time  to  learn  the  JORP  skills  being  taught? 

80.6ygS  17.2  N0 


5.  If  you  had  your  choice,  would  you  lengthen  or  shorten  the  course,  or 
keen  the  course  the  same  length  of  time: 


29.0  Shorten  it . 

11 . 8 Lengthen  it. 

54.9  Keep  it  the  same- 


How  long  should  it  be?_ 
How  long  should  it  be?_ 


6.  Did  the  course  give  you  what  you  expected?  37.6 YES 
What  did  you  expect? 


(days) 

(days) 


57.0  NO 
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18.3  Was  it 


46.2 

□ Better  or 


tJ 


Worse  than  you  expected? 


7.  How  much  difficulty  have  you 
had  with  your  upgrade 
training  in: 


6.4 

10.8 


Completing  COC  materials. 


Completing  OJT  proficiency 
requirements . 


11.7 


Adjusting  your  study  habits 
to  a correspondence  course 
type  of  training 


Almost 

NONE 

A LITTLE 
difficulty 

A LOT  OF 
difficulty 

CANNOT 
DO  IT 

57.0 

31.2 

4.3 

1.1 

65.6 

20.4 

3.2 

0.0 

45.2 

36.5 

5.4 

1.1 

8.  Have  you  completed  your  CDC  requirements?  78.5  yES 
If  YES,  when  did  you  complete  your  CDC? 


21.5 


NO 


18.3 


Before  7 During  1,1  After  taking  the  JORP  t raining 


course. 


9.  How  helpful  do  you  think  the  JORP  course  will  be  or  has  been  in 
completing  your  CDC  requirements? 


6.4 


8.6 

32.3 

15.1 

9.7 

27.9 

Very 

Somewhat 

Did  not 

Of  little 

Of  no  help. 

helpful. 

helpful. 

help,  nor 

help. 

did  not 
hurt. 


10.  Which  parts  of  the  JORP  training  course  were  useful  to  you? 
(Rate  each  part  listed.) 


Parts 

Very 

Useful 

Somewhat 

Useful 

Not  very 
Useful 

A waste 
of  Time 

5.4 

1. 

Narrative  Module 

20.4 

40.9 

19.3 

14.0 

3.2 

2. 

Schematic  (Tables,  Graphs, 

& Figures)  Module 

35.5 

40.9 

8.6 

11.8 

4.3 

3. 

Procedural  Directions  Modul 

e21.5 

37.6 

23.7 

12.9 

5.4 

4. 

AF  and  AFTO  Forms  Module 

38.7 

23.7 

19.3 

12.9 

6.5 

5. 

Imaging  Exercises 

24.7 

31.2 

19.3 

18.3 

4.3 

6. 

Classification  Table  Exer- 
cises 

33.3 

39.8 

9.7 

12.9 

3.2 

7. 

Flow  Chart  Exercises 

33.3 

43.0 

10.8 

9.7 

5.4 

8. 

Tools  for  Learning  Workbook 

30.1 

38.7 

10.8 

15.0 
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12.  Which  instructional  approach  did  you  like  the  BEST? 


40.9  Strand  I 

(Job  Reading  Task  Modules) 

WHY?  (Circle  the  appropriate  one.) 


49.4  Strand  II 

(Imaging,  Flow  Charts,  and 
Classification  Table  Exercises.) 


a.  Easier  e.  Can  be  more  creative  with  the  task. 

b.  Taught  me  new  things.  f.  Other  

c.  Required  less  reading. 

d.  Helps  me  do  my  job  better. 

6 4 13.  Do  you  feel  the  JORP  training  material  was  appropriate  to  your 
AFSC?  40 . 9 yES  52*7NO 


What  is  your  AFSC? 

y 

6.5  14.  Do  you  think  you  could  have  learned  more  by  staying  on  the  job  those 
2-1/2  hours  each  day  rather  than  coming  to  this  course? 

35.5  yes  58*°  NO 


15.  Assume  that  2-1/2  hours  per  day  are  available  for  training  to  in- 
crease your  ability  to  do  your  job.  Rank  order  the  following  in  terms 
of  the  most  useful  way  to  spend  that  2-1/2  hours  of  training  time. 
(Assign  #1  to  the  most  useful,  #2  to  the  next  most  useful,  etc.) 


Rank  Order 


Training  Activity 


JORP  Training  Course 

Increase  Career  Development  Course  (CDC)  Study  Time 

Increased  time  in  Technical  School 

More  Hands-On  Equipment  Time 

General  Reading  Training  Course 

Other  (Specify  and  Rank)  
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16.  Which  way  do  you  BEST  like  to  get  Information  about  something  you 
^ need  to  know  more  about? 

31'^  i.  Read  about  It. 

2.  Listen  to  presentations  or  talks  about  it. 

33.3 

3.  Have  someone  show  and  tell  me  about  It. 

4.  Other  (List)  

17.  How  would  you  compare  yourself  to  9 other  people  on  reading  ability? 

8.6  Better  than  1 

3^2  Better  than  2 

7.5  Better  than  3 

12.9  Better  than  4 
33.3  Better  than  5 

8. 6 Better  than  6 

Better  than  7 

5*^  Better  than  8 


’Better  than  all  9 


18.  At  which  point  do  you  feel  the  JORP  training  Is  most  appropriately  given? 

H.8  «.  , 

3 Prior  to  Basic  Military  Training 

8,6  During  Basic  Military  Training 
21  5 

After  Basic  but  before  Technical  School  or  Directed  Duty  Assignment. 

(DDA) . 

21  5 

Integrated  with  Technical  School  Training. 

Upon  arrival  at  first  PCS  Base. 

11.8lntegrated  with  enrollment  in  the  first  Career  Development  Course 
(CDC). 

3.3  Other  (Soeclfy)  

19.  What  suggestions  would  you  make  to  improve  the  JORP  Program 
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20.  How  confident  are  you  in  your  ability  to  read  and  understand  your 
1.1  job  reading  materials  since  completing  the  JORP  training? 


16.1 

44.1 

36.5 

0.0 

2.2 

Much  more 

Somewhat 

About  the 

Somewhat  less 

Much  less 

confident 

more 

same  as  be- 

confident  than 

confident 

than  before. 

confident. 

fore  the 

before  che 

than  before 

than  before,  course. 

course. 

the  course. 

21.  How  many  other  AF  personnel  do  you  think  would  benefit  from  taking 
3.2  JORP  training? 


16.1 

21.5 

37.7 

21.5 

Nearly 

All 


Most 


Some 


Only  A 
Few 


22.  Do  you  feel  that  it  is  fair  for  the  Air  Force  to  require  people  to 
2 2 participate  in  a program  like -the  JORP? 


12.9 

24.7 

29.0 

20.4 

10.8 

Very 

Somewhat 

All  right 

Somewhat 

Very 

Unfair 

Unfair 

Fair 

Fair 

23.  What  do  you  feel  was  the  objective  of  Strand  I (Job  Reading  Task 
Modules?) 


24.  What  do  you  feel  was  the  objective  of  Strand  IX  (Imaging.  Flow 
Charts,  and  Classification  Table  Exercises? 
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25.  Did  any  of  the  following  things  happen  as  a result  of  taking  the 
JOR?  coarse?  (Check  all  that  apply.) 

NO 


YES 


6.5 

93.5 

6.5 

93.5 

9.7 

90.3 

3.2 

96.8 

11.8 

88.2 

I was  not  released  from  <Juty  to  attend  the  JOR?  course  (I  had 
to  attend  on  my  own  time). 

90*3  ify  shift  was  changed.  What  shift  was  it  changed  to? 


96.8  x had  to  make  up  time  on  the  job  that  I spent  in  JOR?  training. 


38*2  i vas  hassled  by  my  supervisor.  What  happened?^ 


If  you  checked  any  of  the  above,  do  you  think  that  this  decreased 
your  performance  in  the  JORP  course? 

YES  11.8  NO  ?fi,q 

26.  Did  you  complete  a reading  improvement  course  at  Travis  AFB  before 
taking  the  JORP  course? 


5.4 


YES  26.9  NO  67.7 


STUDENT  JORP  QUESTIONS  WITH  COMMENTS 


1.  Why  did  you  like  JORP? 


Helped  me  think 

Helped  me  understand  my  job 

It  got  me  out  of  working 

Helped  my  reading 

It  was  new/different 

Helped  with  forms 

Helped  with  some  things 


3.  Why  has  JORP  helped  on  job? 

Helped  read  & understand  20 

Helped  wxth  forms  5 

Helped  with  TO's  1 

Applies  to  my  job  8 

Widens  base  of  knowledge  1 

Helps  reduce  errors  by 
proper  interpretation 
of  procedures  (a  direct 
quote)  1 

6.  What  did  you  expect  of  course? 

A reading  program  27 

A general  reading  course  4 

A job-specific  reading 


Dislike  JORP? 

Took  up  my  free  time  2 
Wasn't  about  my  job  9 
Didn't  have  time  to  put  it  to  use  1 
Waste  of  time  4 
Course  needs  improving/changing  4 
Was  too  hard  1 
Had  to  pay  for  this  myself  1 
Was  mandatory  3 
Was  treated  like  a child  1 
Test  time  limits  too  short  2 
Didn ' t teach  what  I wanted  2 
Conflicted  with  my  job  1 

Not  helped? 

Not  applicable  to  job  22 
Didn't  teach  anything  new  7 
OJT  is  better  2 
I don't  read  on  job  11 
Not  enough  time  to  learn  well  1 
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How  to  get  things  accomplished/ think  3 


More  learning  1 
Speed  reading  3 
Vocabulary  6 
Spelling  2 
Promotion  1 
College  credit  1 


13 . JORP  material  appropriate  to  AFSC?  Not  appropriate? 


Had  many  things  about  my  job 
Yes,  we  use  books  a lot 
Helped  with  forms 
Methods  will  help  later 
I'm  new  and  need  to  learn 
Somewhat 

Best  for  3-level 


24  Didn't  pertain  to  AFSC  39 

1 We  don't  need  it  for  job  2 

2 We  don't  use  those  forms  1 

1 OJT  is  better  1 

1 There  were  errors  in  mat'l  1 

4 Too  AF  spec if c 1 

1 


14.  Learn  more  on  job:  yes? 

Learn  better  by  OJT 
Course  needs  improvement 
Course  didn't  apply  to  job 
Neither: 

Both  are  useful 

Wasn't  during  job-was  on  my  own  time 


No? 

20  Job  doesn't  teach  this  type  21 
2 of  learning 

^ Learned  to  think  1 

Class  helped  reading  12 

7 Learned  to  keep  notes  in  . 

mind 

2 

Learned  sooner/faster  in  2 
JORP 

Learned  things  for  upgrading  1 
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19.  Suggestions  to  Improve  JORP. 

Add  more  AFSCs  15 
Correct  errors  in  Q's  and  A's  7 
Make  it  voluntary  2 
Lengthen  it  4 
Lenghten  or  eradicate  test  time  limits  2 
Include  spelling  3 
Add  more  instructors/better  ones  2 
More  vocabulary  2 
Plan  time  better  2 
More  tests  and  worksheets  1 
Add  films  1 
Don't  have  it  concurrent  with  CDC  1 
Have  it  deal  with  CDC  1 
Add  fiction  stories  1 
Better  classrooms  1 
More  discussions  2 
Shorten  it  1 
Leave  out  imaging  1 
Leave  out  drawing  1 
Leave  out  parts  that  aren't  useful  1 
Give  a better  understanding  to  students  1 
Get  rid  of  course  6 
"Keep  up  the  good  work!"  1 


2 3 . What  was  the  objective  of  Strand  I ? 

To  teach  student  to  be  able  to  understand  job  tasks  better  through 


creativity,  etc.  49 
To  read  faster  and  better  8 
To  see  if  you  can  r.  -vl  3 
To  see  if  you  can  think  4 
To  help  recall  previous  learning  1 
To  learn  about  different  jobs  1 
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24.  What  was  the  objective  of  Strand  II? 

To  use  tools  or  "tricks"  to  make  reading  or  spoken  information 

more  understandable;  simpler  31 

To  be  able  to  separate,  analyze,  classify,  put  order  to  information  13 

To  be  able  to  find  the  main  idea  to  remember  7 

To  plan  your  job  more  effectively  4 

To  learn  graphs  and  charts  1 

To  learn  to  think,  use  your  imagination  7 

To  find  out  if  a student  can  understand  2 

To  compare  a student's  comprehension  to  that  of  his  peers  1 


25*  What  problems  arose  due  to  JORP  course? 

Had  to  spend  own  time  for  course 
Supervisor  didn't  give  enough  time  off 
Supervisor  said  student  was  getting  out  of  working 
Supervisor  wanted  student  to  work  overtime 
Student  fell  behind  in  work 

Student  felt  peer  pressure  because  of  "dummy  class" 


3 

3 

3 

2 

1 

1 
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APPENDIX  E 

SUPERVISOR'S  JOS-ORIENTED  READING  PROGRAM  SURVEV 

This  appendix  presents  detailed  responses  to  en  ettitudlnel  sur- 
vey of  the  supervisors  of  the  JORP  students.  The  entire  question 
and  its  associated  scale,  and  tha  responses  of  the  sample  are  shown. 

The  total  supervisor  sastple  was  87,  since  six  questionnaires  were  returned 
not  filled  out.  Where  the  question  calls  for  an  open-ended  answer,  re- 
sponses have  been  categorised  and  are  shown  at  the  end  of  this  appendix, 
referenced  by  question  numbers . 


! 
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SUPERVISOR'S  JOB-ORIENTED  READING  PROGRAM  SURVEY 
SUPERVISORS  OF  CLASSES  1-8  N - 87 


Supervises  Airman 

We  ere  trying  to  evaluate  the  effects  of  a new  job-oriented  reading 
training  course.  The  above  airman  was  a recent  graduate  of  this  course. 
We  ask  that  you  fill  out  the  attached  survey  honestly,  and  that  your  com - 
■ante  really  reflect  your  opinions  or  feelings  about  this  airman.  The 
information  you  provide  will  help  us  determine  how  well  the  new  course 
is  meeting  the  needs  of  the  Air  Force  and  will  be  kept  confidential. 

In  accordance  with  paragraph  30,  AFR  12-35,  Air  Force  Privacy  Act 
Program,  the  following  information  about  this  survey  is  provided: 

a.  Authorities 

Title  10  USC,  Section  8012,  Secretary  of  the  Air  Force:  Powers 
and  Duties  Delegation  by;  Executive  Order  9397,  22  Nov  1943, 
Numbering  System  for  Federal  Accounts  Relating  to  Individual 
Persons . 

b.  Principal  Purpose 

This  survey  is  being  conducted  to  obtain  evaluation  data  on  the 
effectiveness  of  Che  Job-Oriented  Reading  Program  (JORP)  devel- 
oped by  the  Human  Resources  Research  Organization  under  contract 
to  the  Air  Force  Human  Resources  Laboratory  at  Lowry  Air  Force 
Base,  Colorado. 

c.  Routine  Use 

The  survey  data  will  be  analyzed  to  provide  specific  recommen- 
dations for  program  changes  in  the  course  prototype.  All  data 
will  be  kept  completely  confidential. 

d.  Disclosure 

Disclosure  is  voluntary.  No  adverse  action  of  any  kind  me* 
be  taken  against  any  individual  who  elects  not  to  participate 
in  the  survey. 


USA7  SCN  7T24 

Sep  1976  thru  Dec  1976 
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Please  answer  the  following  questions. 

1.  How  long  have  you  been  a supervisor?  * -6.7  Years  Months 

2.  How  did  you  feel  about  releasing  this  airman  to  attend  the  Job- 
Oriented  Reading  course  at  the  Base  Education  Office? 


1.2 


Very 

Negative 

WHY? 


18.8 


Somewhat 

Negative 


23.5 


Indifferent 


22.4 


Somewhat 

Positive 


34.1 


Very 
Positive 


3.  How  important  are  reading  skills  for  this  airman  to  perform  satis- 
factorily on  the  job? 


46.0 

41.4 

11.5 

1.1 

Very 

Important 

Relatively 

Not  impor- 

Important 

Unimportant 

tent  At  All 

4.  On  the  average,  how  much  time  during  the  work  day  is  this  airman 
required  to  do  reading  tasks  (such  ea  using  rags  or  manuals,  filling 
out  forms,  studying  CDCs,  etc.)  in  order  to  get  the  job  done? 


* -3^2 Hours 


Minutes 


5.  How  often  do  you  or  others  help  this  airman  with  reading  and 
understanding  written  job  materials? 


15.3 

I 18.8 

18.8 

9.4 

37.7 

Once  a 

3 or  More 

Once  a Vk 

Once  a Mo 

No  Help 

Day 

Times  a Vk 

At  All 
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6.  Did  you  notice  this  alman  having  reading  skill  deficiencies  before 
he/she  ettended  the  Job  Reading  training  course? 

20-7  YES  79-3  NO 

If  YES,  Indicate  all  those  which  apply.  N - 67  responses 
4.7  Airman  could  not  use  Illustrated  parts  breakdowns 
12  >5  Airman  could  not  use  indexes  to  locate  needed  Information 
4 » 7 Alman  could  not  use  grephs  to  get  needed  Information 
10,9  Airman  could  not  use  manuals,  rags 
3'3*  Airman  could  not  use  classification  tables 
Airman  could  not  use  procedural  directions 

17  2 

Airman  repeatedly  asked  for  verbal  instructions  rather 
than  reading  them 

7 8 

* Airman  repeatedly  performed  tasks  Incorrectly  after  reading 
TOs  or  other  Instructions 

4.7  Airman  repeatedly  failed  to  follow  written  Instructions  prior 
to  attempting  s job  task 

15.6  Airman  repeatedly  seemed  not  to  understand  written  Instruc- 
tions for  completing  a task 

1.6  Airman  could  not  read  English 

3<1  Other 

7.  Have  any  of  this  airman’s  reading  skills  been  improved  since  his 
participation  In  the  course? 

PMyBS  15<71no  l66‘h)ldn*t  notice  any  difficulty  before  attending. 
If  YES,  which  sklll(s)?  (See  Question  #6  If  necessary.) 


8.  y°u  sal  this  airman  would  have  learned  more  on  the  job.  If  the 
2 — 1/2  hours  per  day  had  been  spent  there  Instead  of  In  the  course? 

35,0  YES  65,0  NO 

WHY? 
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11.  What  problaa(a)  did  you  hm  la  r«lwilB|  this  atrau  to  attend 
the  eouraaf 

(Hot  All)  . 


Did  you  get  support  from 
for  the  course? 


ir  supervisor  In  releasing  this  airman 


87.1  YES  12.9  NO 


Bow  would  you  coup  are  this  airman  to  nine  othef  airmen  on  reading 
ability? 


5.1  Better  than  1 

5.1  Better  than  2 
12.8  Better  than  3 
10.4  Better  than  4 
19.2  Better  than  5 


ll. s Better  than  6 

14.1  Better  than  7 

14.1  Better  than  8 
7.7  Better  than  9 


14.  Bate  the  overall  effects  of  the  job-oriented  reading  training 
course  on  this  airman’s  job  performance. 


Detrimental  No  Improve- 

to  Perfor-  went 

mance 


Some  Im- 
provement 


Very  Much 
Improved 
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IS.  How  such  supervision 
does  this  airmen 
require  In: 


Doing  assigned  job 
tasks? 

Completing  CDC  know- 
ledge requirements 
(CREs,  VREs,  and  CE)? 

Completing  OJT  pro- 
ficiency require- 
ments (JPGs)? 


Needs 

NO 

Super- 

vision 

Needs  a 

LITTLE 

Super- 

vision 

Needs 

AVERAGE 

Super- 

vision 

Needs  a 
LOT  OP 
Super- 
vision 

Can't 

Rate 

24.4 

38.4 

34.9 

2.3 

0.0 

34.9 

26.5 

20.5 

6.0 

12 

26.2 

35.7 

26.2 

3.6 

8.3 

16.  Race  this  airman's  overall  performance  on  the  job. 


B 

B 

4.8 

28.6 

19.0 

21.4 

23.8 

Unsatis- 

factory 

Marginal 

Below  Avg 

Effective 
& Compe- 
tent 

Above  Avg 

Exception- 
ally Fine 

1 

17-  Which  of  the  following  job  reading  tasks  are  critical  to  the  air 
man's  Job  performance?  Responses  - 253 

9.5  Pse  of  schematics 

24.1  Use  of  procedural  direction 
H.l  Pse  of  narrative  text 

28.1  Use  of  forms 

16.2  y8e  cf  indexes 

-*•  -*  Use  of  graphs 

6*1  Pse  of  classification  tables 
0.4  None 
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18.  Have  you  noticed  job  reading  materials  that  are  particularly  trouble- 
to  moat  airmen?  YES  71.1  NO  29.9 


If  TBS,  Indicate  those  which  you  have  noticed.  40-9  TOs 

^0-9  Manual,  rega 


12.5 


Foma 


5.7 


Letters 


19.  Would  you  expect  the  Job-Oriented  Reading  Program  to  have  any  im- 
pact on  your  unit's  ability  to  get  the  job  done? 

62.7  YES  37^_NO 

Specifically,  what  Impact? 


20.  In  your  experience,  to  what  extent  have  you  found  reading  dlffl- 
cultlea  of  airmen  to  be  a problem  In  completing  upgrade  training 
requirements? 


21.8 

39.1 

32.2 

6.9 

A Great  Deal 

Some 

Very  Little 

None  At  All 
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21.  If  you  had  a choice,  would  you  release  another  airman  to  attend 
the  Job-Oriented  Reading  course? 


33.3 

48.3 

12.6 

5.8 

j 

Definitely 

Probably 

Probably 

Definitely 

YES  YES  NOT  NOT 


22.  Have  you  personally  recommended  other  airmen  for  reading  training 
In  the  past? 

17.2  YES  82.8  HO 

If  YES,  how  many?  ___________ 


23.  Have  you  ever  attended  a reading  Improvement  course  yourself? 

25 • 3 YES  74.7  up 

If  YES,  describe  it.  __________________________________________ 


24.  When  do  you  feel  that  the  job-oriented  reading  training  Is  most 
appropriately  given  to  airman  with  reading  difficulties? 

3.91)00^  know 

6- 5 Prior  to  Basic  Military  Training 
24. 7 During  Basic  Military  Training 

29-8 After  Basic  but  before  Technical  School  or  Direct  IXity  Assign- 
ment (DDA) 

23 Integrated  with  Technical  School  Training 

1,3  Upon  arrival  at  first  PCS  Base 

5 Integrated  with'  enrollment  In  the  first  Career  Development 
Course  (CDC) 

3,9  Other  (Specify) 


I 
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JORP  SUPERVISOR  SURVEY 


QUESTION 

RESPONSES 

N 

2.  How  did  you  feel  about 

Positive 

releasing  this  Airman  to 

Student  needed  reading  training. 

% 

11 

attend  the  JORP  course?  Why 

Self-improvement  is  good. 

11 

Reading  skills  are  important. 

5 

Education  of  any  kind  is  good. 

2 

It  helps  with  CDC. 

Negative 

1 

Student  didn't  need  reading  training. 

8 

Unit  was  understaffed. 

If  student  couldn't  learn  reading  in 

12  years,  he  can't  in  6 weeks. 
(Supervisor  didn't  know  what  JORP 

12 

was  all  about.) 

Indifferent 

Don't  know  if  student  needed  reading 

1 

training. 

2 

Don't  know  about  course  content. 

1 

7.  Which  skills  improved? 

Overall  reading  comprehension. 

19 

Reading  of  instructions. 

1 

Identifying  whole  sentences. 

1 

Speed. 

2 

0.  aiiban  have  learned 

Yes 

more  on  job  than  in  JORP 

Airman  didn't  need  the  course. 

11 

course?  Why? 

Airman  needed  OJT. 

15 
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QUESTION 

RESPONSE 

N 

No 

Airman  needed  reading  training. 

6 

You  can  pick  up  more  (different) 
things)  in  a class. 

1 

15 

| 

Better  training  atmosphere  in 
class. 

10 

1 

| 

Job  reading  is  beneficial  to  all. 

3 

9.  What  areas  of  improvement?  , 

Manuals 

2 

TOs 

3 

Forms  and  Charts 

4 

Regs 

2 

Overall  job  materials 

2 

PDS 

6 

General  reading  skills 

7 

Less  supervision  necessary 

3 

Less  errors 

3 

More  self  confidence 

t 

2 

Better  at  taking  tests 

1 

"Less  talk,  more  reading." 

1 

11.  What  problems  did  you  have 

1 

Understaffed,  loss  of  manhours 

22 

In  releasing  airman  for  course? 

Timing  was  a problem. 

8 

Loss  or  hindrance  of  OJT. 

1 4 

Loss  of  unique  position/duties. 

4 

Airman  took  advantage  of  time  off. 

1 

i 

19.  Would  you  expect  JORP  to 

Positive  Impact 

have  Impact  on  unit? 

If  some  personnel  were  poor  readers. 

5 

Need  RDC  and  COMP  to  work  well. 

20 

Better  understanding  of  manuals. 

12 

ESTION RESPONSES 


